2019-12-09T10:55:12.679 INFO:root:teuthology version: 1.0.0-8b630f9 2019-12-09T10:55:12.680 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.x97DXW.tmp -- --verbose --lock --description rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi --block --owner scheduled_ofriedma@teuthology --archive /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290 2019-12-09T10:55:12.723 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T10:55:12.778 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290 branch: wip-ofriedma-dynamic-disable description: rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: ofriedma@redhat.com first_in_suite: false job_id: '4584290' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-ofriedma-dynamic-disable ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd fast shutdown: false osd objectstore: bluestore osd shutdown pgref assert: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 1351c819e1a9518cde4c8b12d645782009028360 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: notcmalloc sha1: 1351c819e1a9518cde4c8b12d645782009028360 rgw: compression type: random ec-data-pool: true frontend: beast storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-ofriedma-dynamic-disable sha1: 1351c819e1a9518cde4c8b12d645782009028360 owner: scheduled_ofriedma@teuthology priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 1351c819e1a9518cde4c8b12d645782009028360 suite: rgw suite_branch: wip-ofriedma-dynamic-disable suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 1351c819e1a9518cde4c8b12d645782009028360 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.8229 2019-12-09T10:55:12.778 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa; will attempt to use it 2019-12-09T10:55:12.779 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks 2019-12-09T10:55:12.779 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-12-09T10:55:12.781 INFO:teuthology.task.internal:Checking packages... 2019-12-09T10:55:12.795 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '1351c819e1a9518cde4c8b12d645782009028360' 2019-12-09T10:55:12.795 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-12-09T10:55:12.796 INFO:teuthology.packaging:ref: None 2019-12-09T10:55:12.796 INFO:teuthology.packaging:tag: None 2019-12-09T10:55:12.796 INFO:teuthology.packaging:branch: wip-ofriedma-dynamic-disable 2019-12-09T10:55:12.796 INFO:teuthology.packaging:sha1: 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T10:55:12.796 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-ofriedma-dynamic-disable 2019-12-09T10:55:12.950 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7900.g1351c81 2019-12-09T10:55:12.951 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-12-09T10:55:12.960 INFO:teuthology.task.internal:no buildpackages task found 2019-12-09T10:55:12.960 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-12-09T10:55:13.048 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-12-09T10:55:13.048 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T10:55:13.092 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:55:23.108 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:55:33.125 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:55:43.141 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:55:53.155 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:03.172 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:13.187 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:23.203 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:33.220 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:43.227 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:56:53.243 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:03.259 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:13.270 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:23.283 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:33.300 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:43.316 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:57:53.332 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:03.342 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:13.356 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:23.372 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:33.388 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:43.404 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:58:53.419 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:03.428 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:13.444 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:23.462 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:33.475 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:43.490 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T10:59:53.508 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:03.525 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:13.541 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:23.558 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:33.578 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:43.595 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:00:53.612 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:03.628 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:13.640 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:23.655 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:33.665 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:43.678 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:01:53.692 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:03.705 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:13.722 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:23.740 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:33.756 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:43.768 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:02:53.777 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:03.791 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:13.803 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:23.814 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:33.825 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:43.835 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-12-09T11:03:53.853 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_ofriedma@teuthology', 'description': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290', 'machine_type': 'smithi'} 2019-12-09T11:03:53.868 DEBUG:teuthology.lock.ops:locked ubuntu@smithi174.front.sepia.ceph.com, ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:03:53.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T11:03:54.115 INFO:teuthology.provision.fog.smithi106:Scheduling deploy of centos 7.6 2019-12-09T11:03:54.118 INFO:teuthology.provision.fog.smithi174:Scheduling deploy of centos 7.6 2019-12-09T11:03:54.430 INFO:teuthology.orchestra.console:Power off smithi174 2019-12-09T11:03:54.431 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T11:03:54.453 INFO:teuthology.orchestra.console:Power off smithi106 2019-12-09T11:03:54.453 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-12-09T11:03:54.462 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:54.475 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:54.585 INFO:teuthology.orchestra.console:Power off for smithi174 completed 2019-12-09T11:03:54.597 INFO:teuthology.orchestra.console:Power off for smithi106 completed 2019-12-09T11:03:54.687 INFO:teuthology.orchestra.console:Power on smithi174 2019-12-09T11:03:54.687 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T11:03:54.698 INFO:teuthology.orchestra.console:Power on smithi106 2019-12-09T11:03:54.698 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-12-09T11:03:54.710 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:54.720 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:58.736 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:58.745 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:03:58.955 INFO:teuthology.orchestra.console:Power on for smithi106 completed 2019-12-09T11:03:59.056 INFO:teuthology.provision.fog.smithi106:Waiting for deploy to finish 2019-12-09T11:04:02.848 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi174.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-12-09T11:04:03.059 INFO:teuthology.orchestra.console:Power on for smithi174 completed 2019-12-09T11:04:03.160 INFO:teuthology.provision.fog.smithi174:Waiting for deploy to finish 2019-12-09T11:06:34.140 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:06:43.214 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:06:52.269 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:00.239 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:01.326 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:09.293 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:10.378 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:18.353 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:19.433 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:27.437 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:28.490 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:36.494 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:45.549 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:07:54.606 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:08:03.661 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:08:34.514 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:08:34.722 INFO:teuthology.orchestra.run.smithi174:> hostname 2019-12-09T11:08:35.290 INFO:teuthology.orchestra.run.smithi174.stdout:smithi174 2019-12-09T11:08:35.290 INFO:teuthology.orchestra.run.smithi174:> grep smithi174 /etc/hosts 2019-12-09T11:08:35.374 INFO:teuthology.orchestra.run.smithi174.stdout:172.21.15.174 smithi174.front.sepia.ceph.com smithi174 2019-12-09T11:08:35.375 INFO:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2019-12-09T11:08:35.522 INFO:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174/smithi174/g' /etc/hosts 2019-12-09T11:08:35.630 INFO:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/172.21.15.174/172.21.15.174/g' /etc/hosts 2019-12-09T11:08:35.725 INFO:teuthology.orchestra.run.smithi174:> sudo sed -i -e 's/smithi174/smithi174/g' /etc/hostname 2019-12-09T11:08:35.804 INFO:teuthology.orchestra.run.smithi174:> sudo hostname smithi174 2019-12-09T11:08:35.874 INFO:teuthology.provision.fog.smithi174:Deploy complete! 2019-12-09T11:08:35.876 DEBUG:teuthology.parallel:result is None 2019-12-09T11:09:09.692 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:09:09.916 INFO:teuthology.orchestra.run.smithi106:> hostname 2019-12-09T11:09:10.490 INFO:teuthology.orchestra.run.smithi106.stdout:smithi106.front.sepia.ceph.com 2019-12-09T11:09:10.491 INFO:teuthology.orchestra.run.smithi106:> grep smithi106.front.sepia.ceph.com /etc/hosts 2019-12-09T11:09:10.566 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.15.106 smithi106.front.sepia.ceph.com smithi106 2019-12-09T11:09:10.566 INFO:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2019-12-09T11:09:10.703 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hosts 2019-12-09T11:09:10.807 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/172.21.15.106/172.21.15.106/g' /etc/hosts 2019-12-09T11:09:10.902 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i -e 's/smithi106.front.sepia.ceph.com/smithi106/g' /etc/hostname 2019-12-09T11:09:10.987 INFO:teuthology.orchestra.run.smithi106:> sudo hostname smithi106 2019-12-09T11:09:11.057 INFO:teuthology.provision.fog.smithi106:Deploy complete! 2019-12-09T11:09:11.058 DEBUG:teuthology.parallel:result is None 2019-12-09T11:09:11.224 INFO:teuthology.lock.ops:Checking smithi174.front.sepia.ceph.com 2019-12-09T11:09:11.225 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T11:09:11.241 INFO:teuthology.lock.ops:Checking smithi106.front.sepia.ceph.com 2019-12-09T11:09:11.241 INFO:teuthology.lock.ops:New key found. Updating... 2019-12-09T11:09:11.255 INFO:teuthology.lock.ops:Updating [smithi174.front.sepia.ceph.com]: set os type and version on server 2019-12-09T11:09:11.255 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:09:11.256 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:09:11.347 INFO:teuthology.orchestra.run.smithi174:> true 2019-12-09T11:09:11.708 INFO:teuthology.orchestra.run.smithi174:> uname -m 2019-12-09T11:09:11.767 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2019-12-09T11:09:11.767 INFO:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2019-12-09T11:09:11.818 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux" 2019-12-09T11:09:11.818 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="7 (Core)" 2019-12-09T11:09:11.818 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="7" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T11:09:11.819 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T11:09:11.820 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T11:09:11.821 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:11.821 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2019-12-09T11:09:11.835 INFO:teuthology.lock.ops:Updating [smithi106.front.sepia.ceph.com]: set os type and version on server 2019-12-09T11:09:11.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:09:11.836 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:09:11.925 INFO:teuthology.orchestra.run.smithi106:> true 2019-12-09T11:09:12.307 INFO:teuthology.orchestra.run.smithi106:> uname -m 2019-12-09T11:09:12.367 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2019-12-09T11:09:12.368 INFO:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2019-12-09T11:09:12.424 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Linux" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="7 (Core)" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="7" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T11:09:12.425 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T11:09:12.426 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T11:09:12.427 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:12.427 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2019-12-09T11:09:12.438 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-12-09T11:09:12.465 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi106.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC4Y2hUa3UcyVVQvDv9tlcLB8dL8kTGA/vbCDskZ4q/lbHQ8OUNeybWaTIkW+Eg8mtVWcfotE9eM3CCYVs43vbgoGx7Nvl0jFLxmSc42NxzateYVE+CCB98xA+4pCwxbZ476SNaVOl/PcloPG3xwA7UIx2i0UwRT37H8OUwzGB3YuYLCp0tJZTf1k9s35T+OJTJMnCEDlghC3ayLtIIhtQkaW0QG4Xloqs6ER99D8sCmEr4lGoSJ1yxNAjTdHosHzgCeVn1MRfqfwE9nxpULaO0w9RskLkCllUUlAPgTGPHv4NlQ+5g6JMADcDhHNdpdAwaefyU2LCgEhukij4vT1H smithi174.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3DSmS5rlGuNxmSFQRAnHMrHbZg4ztgE7CYu35NfteOEMP1P0f/aPvaQri8Lkr1ts1zs2LmyOBxvXli5VP91y2tA7kTrCc6SB15TqzIQrMccq6H6uxA8g6KyBrAWTimXGoxpa4n91ZYxhdI67Fuz/j/OqRxHnOpVLedv70RWRwNKm0xL6cZLuIlBibb1zcvT5QiwE4FFug3YNBzZEKBQU15weGHYEjPpjy+lVQm3BfJ6QOfqWiiZiC84oPvCtepurT4X3Mr/wXqkcCspE00PSVLNmHMOF2tGgdKeDgwgROvKOg1juiXIqAjkjA3GV4RaIQBirkPDXSzxreGANcExnn 2019-12-09T11:09:12.466 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-12-09T11:09:12.506 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-12-09T11:09:12.511 INFO:teuthology.task.internal:Saving configuration 2019-12-09T11:09:12.529 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-12-09T11:09:12.537 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-12-09T11:09:12.550 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 11:03:53.858936', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi174.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDp+aWRkT6lwTzkV1YueC6hepdumbnKJT60t6YQr4kEAPAMGqS4iYw00yI1ihzvWpyNav0/ZFGuiycJnOdZronk=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290'} 2019-12-09T11:09:12.563 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-12-09 11:03:53.858955', u'locked_by': u'scheduled_ofriedma@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi106.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJNqVZ08kOIyKmVMJDhrMuZDs3VZymKvAm3C46SYfjHz9AtUy/xcxBjLoCjso54cDFvaFeX6pRMY6q/nijMpOD0=', u'description': u'/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290'} 2019-12-09T11:09:12.563 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-12-09T11:09:12.587 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T11:09:12.588 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T11:09:12.588 INFO:teuthology.run_tasks:Running task console_log... 2019-12-09T11:09:12.692 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-12-09T11:09:12.693 INFO:teuthology.run_tasks:Running task internal.connect... 2019-12-09T11:09:12.701 INFO:teuthology.task.internal:Opening connections... 2019-12-09T11:09:12.701 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:09:12.702 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:09:12.796 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:09:12.796 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:09:12.890 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-12-09T11:09:12.897 INFO:teuthology.orchestra.run.smithi106:> uname -m 2019-12-09T11:09:13.090 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2019-12-09T11:09:13.091 INFO:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2019-12-09T11:09:13.141 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Linux" 2019-12-09T11:09:13.141 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="7 (Core)" 2019-12-09T11:09:13.141 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2019-12-09T11:09:13.141 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="7" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T11:09:13.142 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T11:09:13.143 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:13.144 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2019-12-09T11:09:13.164 INFO:teuthology.orchestra.run.smithi174:> uname -m 2019-12-09T11:09:13.255 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2019-12-09T11:09:13.256 INFO:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2019-12-09T11:09:13.309 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Linux" 2019-12-09T11:09:13.309 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="7 (Core)" 2019-12-09T11:09:13.309 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2019-12-09T11:09:13.309 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="7" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://www.centos.org/" 2019-12-09T11:09:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-12-09T11:09:13.311 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:13.312 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2019-12-09T11:09:13.324 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-12-09T11:09:13.361 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-12-09T11:09:13.372 INFO:teuthology.task.internal:Checking for old test directory... 2019-12-09T11:09:13.373 INFO:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2019-12-09T11:09:13.376 INFO:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2019-12-09T11:09:13.458 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-12-09T11:09:13.468 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-12-09T11:09:13.469 INFO:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2019-12-09T11:09:13.472 INFO:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2019-12-09T11:09:13.559 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-12-09T11:09:13.595 INFO:teuthology.run_tasks:Running task kernel... 2019-12-09T11:09:13.612 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}} 2019-12-09T11:09:13.612 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-12-09T11:09:13.612 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:09:13.613 INFO:teuthology.orchestra.run.smithi106:> sudo package-cleanup -y --oldkernels 2019-12-09T11:09:14.492 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:09:15.472 INFO:teuthology.orchestra.run.smithi106.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:15.472 INFO:teuthology.orchestra.run.smithi106.stdout:Trying other mirror. 2019-12-09T11:09:15.472 INFO:teuthology.orchestra.run.smithi106.stdout:To address this issue please refer to the below wiki article 2019-12-09T11:09:15.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:15.473 INFO:teuthology.orchestra.run.smithi106.stdout:https://wiki.centos.org/yum-errors 2019-12-09T11:09:15.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:15.473 INFO:teuthology.orchestra.run.smithi106.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T11:09:15.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:09:15.625 INFO:teuthology.orchestra.run.smithi106.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:15.625 INFO:teuthology.orchestra.run.smithi106.stdout:Trying other mirror. 2019-12-09T11:09:15.717 INFO:teuthology.orchestra.run.smithi106.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:15.717 INFO:teuthology.orchestra.run.smithi106.stdout:Trying other mirror. 2019-12-09T11:09:15.980 INFO:teuthology.orchestra.run.smithi106.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:15.981 INFO:teuthology.orchestra.run.smithi106.stdout:Trying other mirror. 2019-12-09T11:09:16.133 INFO:teuthology.orchestra.run.smithi106.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:16.134 INFO:teuthology.orchestra.run.smithi106.stdout:Trying other mirror. 2019-12-09T11:09:16.657 INFO:teuthology.orchestra.run.smithi106.stdout:No old kernels to remove 2019-12-09T11:09:16.672 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:09:16.673 INFO:teuthology.orchestra.run.smithi174:> sudo package-cleanup -y --oldkernels 2019-12-09T11:09:17.558 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:09:18.493 INFO:teuthology.orchestra.run.smithi174.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout:Trying other mirror. 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout:To address this issue please refer to the below wiki article 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout:https://wiki.centos.org/yum-errors 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-12-09T11:09:18.494 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:18.658 INFO:teuthology.orchestra.run.smithi174.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:18.658 INFO:teuthology.orchestra.run.smithi174.stdout:Trying other mirror. 2019-12-09T11:09:18.750 INFO:teuthology.orchestra.run.smithi174.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:18.750 INFO:teuthology.orchestra.run.smithi174.stdout:Trying other mirror. 2019-12-09T11:09:18.893 INFO:teuthology.orchestra.run.smithi174.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:18.893 INFO:teuthology.orchestra.run.smithi174.stdout:Trying other mirror. 2019-12-09T11:09:19.045 INFO:teuthology.orchestra.run.smithi174.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/d0f1134f0f9a3fed89fb7177d07a42aa429e0e37c0d6b3e1c907d638a5cd1164-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-12-09T11:09:19.046 INFO:teuthology.orchestra.run.smithi174.stdout:Trying other mirror. 2019-12-09T11:09:19.624 INFO:teuthology.orchestra.run.smithi174.stdout:No old kernels to remove 2019-12-09T11:09:19.640 INFO:teuthology.orchestra.run.smithi174:> uname -r 2019-12-09T11:09:19.727 INFO:teuthology.orchestra.run.smithi174.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:09:19.727 INFO:teuthology.task.kernel:Running kernel on smithi174: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:09:19.728 INFO:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2019-12-09T11:09:19.972 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:09:20.013 INFO:teuthology.orchestra.run.smithi174.stdout:Determining fastest mirrors 2019-12-09T11:09:20.145 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:09:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T11:09:20.146 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:09:20.147 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:09:21.723 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:09:21.724 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:09:21.724 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T11:09:23.002 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:09:23.848 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:23.848 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:09:23.850 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T11:09:23.851 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:23.851 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:09:23.851 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:09:23.851 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T11:09:23.852 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:09:23.852 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 50 M 2019-12-09T11:09:23.852 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 64 M 2019-12-09T11:09:23.852 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:09:23.857 INFO:teuthology.orchestra.run.smithi174.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T11:09:25.110 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:09:25.294 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:09:26.204 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:09:26.204 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:10:01.156 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:10:01.653 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:10:01.702 INFO:teuthology.orchestra.run.smithi174:> rpm -q kernel --last | head -n 1 2019-12-09T11:10:02.026 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 11:09:28 AM UTC 2019-12-09T11:10:02.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:02.028 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:02.028 INFO:teuthology.orchestra.run.smithi106:> uname -r 2019-12-09T11:10:02.124 INFO:teuthology.orchestra.run.smithi106.stdout:3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:10:02.125 INFO:teuthology.task.kernel:Running kernel on smithi106: 3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:10:02.125 INFO:teuthology.orchestra.run.smithi106:> sudo yum install -y kernel 2019-12-09T11:10:02.388 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:10:02.433 INFO:teuthology.orchestra.run.smithi106.stdout:Determining fastest mirrors 2019-12-09T11:10:02.568 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:10:02.569 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: ftp.linux.ncsu.edu 2019-12-09T11:10:02.570 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:10:02.570 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:10:04.228 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:10:04.228 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:10:04.228 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package kernel.x86_64 0:3.10.0-1062.9.1.el7 will be installed 2019-12-09T11:10:05.550 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:10:06.558 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:06.559 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:10:06.562 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:06.562 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:10:06.562 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:10:06.562 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:10:06.562 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout: kernel x86_64 3.10.0-1062.9.1.el7 updates 50 M 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2019-12-09T11:10:06.563 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:06.564 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 50 M 2019-12-09T11:10:06.564 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 64 M 2019-12-09T11:10:06.564 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:10:06.568 INFO:teuthology.orchestra.run.smithi106.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T11:10:07.557 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:10:07.751 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:10:08.586 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:10:08.586 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:10:44.187 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T11:10:45.064 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : kernel-3.10.0-1062.9.1.el7.x86_64 1/1 2019-12-09T11:10:45.064 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:45.065 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:10:45.065 INFO:teuthology.orchestra.run.smithi106.stdout: kernel.x86_64 0:3.10.0-1062.9.1.el7 2019-12-09T11:10:45.065 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:45.065 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:10:45.118 INFO:teuthology.orchestra.run.smithi106:> rpm -q kernel --last | head -n 1 2019-12-09T11:10:45.522 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 11:10:11 AM UTC 2019-12-09T11:10:45.523 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:45.523 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:45.523 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-12-09T11:10:45.523 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T11:10:45.523 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-12-09T11:10:45.523 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-12-09T11:10:45.523 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T11:10:45.524 INFO:teuthology.orchestra.run.smithi174:> rpm -q kernel --last | head -n 1 2019-12-09T11:10:45.902 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 11:09:28 AM UTC 2019-12-09T11:10:45.904 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:45.905 INFO:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2 2019-12-09T11:10:46.019 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2 2019-12-09T11:10:46.019 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.02 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 0.76.el7.centos 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Group : System Environment/Base 2019-12-09T11:10:46.020 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 0 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Relocations : (not relocatable) 2019-12-09T11:10:46.021 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : CentOS BuildSystem 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T11:10:46.022 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T11:10:46.023 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2019-12-09T11:10:46.024 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T11:10:46.024 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:46.024 INFO:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T11:10:46.494 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2019-12-09T11:10:46.629 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:46.644 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T11:10:46.892 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:10:46.909 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T11:10:46.940 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T11:10:46.956 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T11:10:46.982 INFO:teuthology.orchestra.run.smithi174.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T11:10:46.991 INFO:teuthology.orchestra.run.smithi174.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T11:10:48.133 INFO:teuthology.orchestra.run.smithi174.stderr:done 2019-12-09T11:10:48.134 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:10:48.260 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.zsss7CZjeD 2019-12-09T11:10:48.261 INFO:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.zsss7CZjeD 2019-12-09T11:10:48.374 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.zsss7CZjeD 2019-12-09T11:10:48.570 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.zsss7CZjeD is 6KB 2019-12-09T11:10:48.639 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.zsss7CZjeD 2019-12-09T11:10:48.695 INFO:teuthology.orchestra.run.smithi174:> sudo grub2-set-default 0 2019-12-09T11:10:48.848 INFO:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2019-12-09T11:10:48.851 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-12-09T11:10:48.851 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=None, version=distro) 2019-12-09T11:10:48.852 INFO:teuthology.orchestra.run.smithi106:> rpm -q kernel --last | head -n 1 2019-12-09T11:10:49.215 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-3.10.0-1062.9.1.el7.x86_64 Mon 09 Dec 2019 11:10:11 AM UTC 2019-12-09T11:10:49.217 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:49.217 INFO:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2 2019-12-09T11:10:49.359 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2 2019-12-09T11:10:49.359 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2019-12-09T11:10:49.359 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.02 2019-12-09T11:10:49.359 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 0.76.el7.centos 2019-12-09T11:10:49.359 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Group : System Environment/Base 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 0 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-12-09T11:10:49.360 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : kbuilder.bsys.centos.org 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:Relocations : (not relocatable) 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : CentOS BuildSystem 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2019-12-09T11:10:49.361 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2019-12-09T11:10:49.362 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2019-12-09T11:10:49.363 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-12-09T11:10:49.363 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:49.363 INFO:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-12-09T11:10:49.883 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2019-12-09T11:10:50.019 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:10:50.034 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.9.1.el7.x86_64.img 2019-12-09T11:10:50.334 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-12-09T11:10:50.358 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-12-09T11:10:50.391 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-12-09T11:10:50.409 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-12-09T11:10:50.437 INFO:teuthology.orchestra.run.smithi106.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-12-09T11:10:50.447 INFO:teuthology.orchestra.run.smithi106.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-12-09T11:10:51.621 INFO:teuthology.orchestra.run.smithi106.stderr:done 2019-12-09T11:10:51.623 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:10:51.679 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.xVGmWXkJnu 2019-12-09T11:10:51.679 INFO:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xVGmWXkJnu 2019-12-09T11:10:51.742 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.xVGmWXkJnu 2019-12-09T11:10:51.872 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.xVGmWXkJnu is 6KB 2019-12-09T11:10:51.922 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.xVGmWXkJnu 2019-12-09T11:10:51.965 INFO:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 0 2019-12-09T11:10:52.113 INFO:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2019-12-09T11:10:52.116 INFO:teuthology.misc:Re-opening connections... 2019-12-09T11:10:52.116 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:10:52.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:10:52.118 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:10:52.199 INFO:teuthology.orchestra.run.smithi106:> true 2019-12-09T11:10:52.495 DEBUG:teuthology.misc:waited 0.378644943237 2019-12-09T11:10:53.495 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:10:53.496 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:10:53.496 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:11:53.498 DEBUG:teuthology.orchestra.remote:timed out 2019-12-09T11:11:53.498 DEBUG:teuthology.misc:waited 61.3824729919 2019-12-09T11:11:54.499 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:11:54.499 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:11:54.500 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:26.041 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2019-12-09T11:12:26.041 DEBUG:teuthology.misc:waited 93.9249157906 2019-12-09T11:12:27.042 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:12:27.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:27.043 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:27.044 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2019-12-09T11:12:27.044 DEBUG:teuthology.misc:waited 94.9279727936 2019-12-09T11:12:28.045 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:12:28.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:28.046 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:28.047 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2019-12-09T11:12:28.047 DEBUG:teuthology.misc:waited 95.9307188988 2019-12-09T11:12:29.047 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:12:29.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:29.048 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:29.049 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.174 2019-12-09T11:12:29.049 DEBUG:teuthology.misc:waited 96.9334499836 2019-12-09T11:12:30.051 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:12:30.051 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:30.051 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:30.127 INFO:teuthology.orchestra.run.smithi174:> true 2019-12-09T11:12:30.440 DEBUG:teuthology.misc:waited 98.3237199783 2019-12-09T11:12:31.441 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-12-09T11:12:31.441 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T11:12:31.441 INFO:teuthology.orchestra.run.smithi174:> uname -r 2019-12-09T11:12:31.518 INFO:teuthology.orchestra.run.smithi174.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:12:31.518 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:12:31.518 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T11:12:31.519 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T11:12:31.519 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T11:12:31.519 INFO:teuthology.orchestra.run.smithi106:> uname -r 2019-12-09T11:12:52.413 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2019-12-09T11:12:52.413 ERROR:teuthology.task.kernel:Saw exception Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 695, in wait_for_reboot assert not need_to_install(ctx, client, need_install[client]), \ File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/kernel.py", line 184, in need_to_install stdout=uname_fp, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/cluster.py", line 64, in run return [remote.run(**kwargs) for remote in remotes] File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 198, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 428, in run r.execute() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 98, in execute self.client.exec_command(self.command) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/client.py", line 493, in exec_command chan = self._transport.open_session(timeout=timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 806, in open_session timeout=timeout, File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/local/lib/python2.7/site-packages/paramiko/transport.py", line 933, in open_channel raise e error: [Errno 104] Connection reset by peer 2019-12-09T11:12:53.416 INFO:teuthology.misc:Re-opening connections... 2019-12-09T11:12:53.417 INFO:teuthology.misc:trying to connect to ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:12:53.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:53.417 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:53.487 INFO:teuthology.orchestra.run.smithi106:> true 2019-12-09T11:12:53.769 DEBUG:teuthology.misc:waited 0.352099895477 2019-12-09T11:12:54.770 INFO:teuthology.misc:trying to connect to ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:12:54.771 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:12:54.772 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:12:54.867 INFO:teuthology.orchestra.run.smithi174:> true 2019-12-09T11:12:55.265 DEBUG:teuthology.misc:waited 1.84856510162 2019-12-09T11:12:56.266 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-12-09T11:12:56.267 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.9.1.el7.x86_64"... 2019-12-09T11:12:56.267 INFO:teuthology.orchestra.run.smithi106:> uname -r 2019-12-09T11:12:56.325 INFO:teuthology.orchestra.run.smithi106.stdout:3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:12:56.325 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.9.1.el7.x86_64 2019-12-09T11:12:56.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-12-09T11:12:57.326 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-12-09T11:12:57.326 INFO:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T11:12:57.479 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2019-12-09T11:12:57.492 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-12-09T11:12:57.492 INFO:teuthology.orchestra.run.smithi106:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-12-09T11:12:57.608 INFO:teuthology.orchestra.run.smithi106.stdout:ttyS1 2019-12-09T11:12:57.619 INFO:teuthology.run_tasks:Running task internal.base... 2019-12-09T11:12:57.632 INFO:teuthology.task.internal:Creating test directory... 2019-12-09T11:12:57.633 INFO:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T11:12:57.636 INFO:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-12-09T11:12:57.723 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-12-09T11:12:57.733 INFO:teuthology.run_tasks:Running task internal.archive... 2019-12-09T11:12:57.746 INFO:teuthology.task.internal:Creating archive directory... 2019-12-09T11:12:57.746 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T11:12:57.749 INFO:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-12-09T11:12:57.849 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-12-09T11:12:57.859 INFO:teuthology.task.internal:Enabling coredump saving... 2019-12-09T11:12:57.859 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T11:12:57.863 INFO:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T11:12:57.952 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T11:12:57.962 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-12-09T11:12:57.964 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-12-09T11:12:57.974 INFO:teuthology.task.internal:Configuring sudo... 2019-12-09T11:12:57.975 INFO:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T11:12:57.977 INFO:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-12-09T11:12:58.066 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-12-09T11:12:58.078 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-12-09T11:12:58.078 INFO:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T11:12:58.082 INFO:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-12-09T11:12:58.158 INFO:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T11:12:58.231 INFO:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T11:12:58.311 INFO:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T11:12:58.368 INFO:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T11:12:58.467 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T11:12:58.530 INFO:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T11:12:58.638 INFO:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-12-09T11:12:58.759 INFO:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T11:12:58.859 INFO:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-12-09T11:12:58.976 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-12-09T11:12:59.075 INFO:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2019-12-09T11:12:59.078 INFO:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2019-12-09T11:12:59.192 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T11:12:59.198 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-12-09T11:12:59.245 INFO:teuthology.run_tasks:Running task internal.timer... 2019-12-09T11:12:59.255 INFO:teuthology.task.internal:Starting timer... 2019-12-09T11:12:59.255 INFO:teuthology.run_tasks:Running task pcp... 2019-12-09T11:12:59.288 INFO:teuthology.run_tasks:Running task selinux... 2019-12-09T11:12:59.301 INFO:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2019-12-09T11:12:59.416 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: [ OK ] 2019-12-09T11:12:59.418 INFO:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2019-12-09T11:12:59.572 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: [ OK ] 2019-12-09T11:12:59.574 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-12-09T11:12:59.575 INFO:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2019-12-09T11:12:59.661 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2019-12-09T11:12:59.661 INFO:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2019-12-09T11:12:59.753 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2019-12-09T11:12:59.753 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi174.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive'} 2019-12-09T11:12:59.754 INFO:teuthology.orchestra.run.smithi106:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T11:12:59.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T11:12:59.857 INFO:teuthology.orchestra.run.smithi174:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-12-09T11:12:59.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T11:12:59.971 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-12-09T11:12:59.971 INFO:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2019-12-09T11:13:00.090 INFO:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2019-12-09T11:13:00.218 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-12-09T11:13:00.228 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-12-09T11:13:00.229 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-12-09T11:13:00.244 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-12-09T11:13:00.247 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-12-09T11:13:00.611 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-12-09T11:13:01.802 INFO:teuthology.task.ansible.out: [WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2019-12-09T11:13:04.427 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-12-09T11:13:04.451 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:04 +0000 (0:00:00.066) 0:00:00.066 ******* 2019-12-09T11:13:04.463 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:04 +0000 (0:00:00.012) 0:00:00.078 ******* 2019-12-09T11:13:07.658 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi106.front.sepia.ceph.com] 2019-12-09T11:13:07.735 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:07 +0000 (0:00:03.271) 0:00:03.350 ******* 2019-12-09T11:13:07.810 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] 2019-12-09T11:13:07.885 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:07 +0000 (0:00:00.149) 0:00:03.500 ******* 2019-12-09T11:13:08.194 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-12-09T11:13:08.195 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T11:13:08.230 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.344) 0:00:03.844 ******* 2019-12-09T11:13:08.236 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T11:13:08.273 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.042) 0:00:03.888 ******* 2019-12-09T11:13:08.796 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-12-09T11:13:08.797 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T11:13:08.833 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.559) 0:00:04.447 ******* 2019-12-09T11:13:08.849 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-12-09T11:13:08.886 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.053) 0:00:04.501 ******* 2019-12-09T11:13:08.892 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-12-09T11:13:08.893 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:08.928 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.042) 0:00:04.543 ******* 2019-12-09T11:13:08.934 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:08.970 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:08 +0000 (0:00:00.041) 0:00:04.585 ******* 2019-12-09T11:13:09.369 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-12-09T11:13:09.370 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:13:09.407 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:09 +0000 (0:00:00.436) 0:00:05.022 ******* 2019-12-09T11:13:09.413 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:13:09.453 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:09 +0000 (0:00:00.045) 0:00:05.067 ******* 2019-12-09T11:13:10.860 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2019-12-09T11:13:10.861 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575889989.42-226557438908518/source", "state": "file", "uid": 0} 2019-12-09T11:13:10.863 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575889989.47-242099216764070/source", "state": "file", "uid": 0} 2019-12-09T11:13:10.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:10 +0000 (0:00:01.453) 0:00:06.521 ******* 2019-12-09T11:13:10.946 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:10 +0000 (0:00:00.039) 0:00:06.561 ******* 2019-12-09T11:13:11.457 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2019-12-09T11:13:11.459 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:11.460 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:11.724 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:11.766 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.033 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.044 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.340 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.382 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.632 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.728 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:12.959 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:13.015 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:13.286 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:13.333 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-12-09T11:13:13.481 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** 2019-12-09T11:13:13.482 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2019-12-09T11:13:13.489 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-12-09T11:13:13.679 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:13 +0000 (0:00:02.732) 0:00:09.294 ******* 2019-12-09T11:13:13.714 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:13 +0000 (0:00:00.034) 0:00:09.329 ******* 2019-12-09T11:13:14.603 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2019-12-09T11:13:14.605 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] 2019-12-09T11:13:14.618 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] 2019-12-09T11:13:14.729 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-12-09T11:13:14.730 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:14 +0000 (0:00:01.015) 0:00:10.344 ******* 2019-12-09T11:13:14.769 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:14 +0000 (0:00:00.039) 0:00:10.384 ******* 2019-12-09T11:13:15.244 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-12-09T11:13:15.246 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.233324", "end": "2019-12-09 11:13:15.219977", "rc": 0, "start": "2019-12-09 11:13:14.986653", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T11:13:15.286 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.517) 0:00:10.901 ******* 2019-12-09T11:13:15.295 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.241757", "end": "2019-12-09 11:13:15.231502", "rc": 0, "start": "2019-12-09 11:13:14.989745", "stderr": "", "stderr_lines": [], "stdout": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "stdout_lines": ["2c3c6d863a854d6a1e155fc475e8273d88b91491"]} 2019-12-09T11:13:15.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.053) 0:00:10.955 ******* 2019-12-09T11:13:15.597 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T11:13:15.597 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.007977", "end": "2019-12-09 11:13:15.571266", "failed_when_result": false, "rc": 0, "start": "2019-12-09 11:13:15.563289", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T11:13:15.642 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.301) 0:00:11.257 ******* 2019-12-09T11:13:15.669 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T11:13:15.671 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.067) 0:00:11.324 ******* 2019-12-09T11:13:15.716 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-12-09T11:13:15.717 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.027315", "end": "2019-12-09 11:13:15.648041", "failed_when_result": false, "rc": 0, "start": "2019-12-09 11:13:15.620726", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-12-09T11:13:15.762 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.052) 0:00:11.377 ******* 2019-12-09T11:13:15.770 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T11:13:15.771 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.811 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.049) 0:00:11.426 ******* 2019-12-09T11:13:15.818 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-12-09T11:13:15.820 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.858 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.046) 0:00:11.473 ******* 2019-12-09T11:13:15.860 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:15.866 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T11:13:15.867 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.907 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.049) 0:00:11.522 ******* 2019-12-09T11:13:15.914 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-12-09T11:13:15.914 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.953 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.046) 0:00:11.568 ******* 2019-12-09T11:13:15.958 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T11:13:15.959 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:15.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:15 +0000 (0:00:00.044) 0:00:11.613 ******* 2019-12-09T11:13:16.004 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-12-09T11:13:16.005 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:16.042 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.043) 0:00:11.657 ******* 2019-12-09T11:13:16.047 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T11:13:16.048 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:16.085 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.043) 0:00:11.700 ******* 2019-12-09T11:13:16.092 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-12-09T11:13:16.094 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:16.130 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.044) 0:00:11.745 ******* 2019-12-09T11:13:16.156 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-12-09T11:13:16.158 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:16.194 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.063) 0:00:11.809 ******* 2019-12-09T11:13:16.407 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-12-09T11:13:16.409 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:13:16.447 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.252) 0:00:12.062 ******* 2019-12-09T11:13:16.505 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:13:16.542 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:16 +0000 (0:00:00.095) 0:00:12.157 ******* 2019-12-09T11:13:16.797 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-12-09T11:13:16.797 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T11:13:16.923 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-12-09T11:13:17.081 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T11:13:17.260 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-12-09T11:13:17.428 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T11:13:17.596 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-12-09T11:13:17.746 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T11:13:17.934 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-12-09T11:13:18.062 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T11:13:18.241 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-12-09T11:13:18.308 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T11:13:18.561 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-12-09T11:13:18.646 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T11:13:18.876 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-12-09T11:13:18.981 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T11:13:19.222 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-12-09T11:13:19.327 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T11:13:19.559 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-12-09T11:13:19.646 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T11:13:19.878 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-12-09T11:13:19.987 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T11:13:20.219 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-12-09T11:13:20.335 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T11:13:20.550 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-12-09T11:13:20.677 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T11:13:20.887 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-12-09T11:13:21.024 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T11:13:21.246 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-12-09T11:13:21.321 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T11:13:21.585 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T11:13:21.597 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-12-09T11:13:21.924 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T11:13:21.947 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-12-09T11:13:22.242 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T11:13:22.274 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-12-09T11:13:22.538 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T11:13:22.636 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-12-09T11:13:22.869 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T11:13:22.969 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-12-09T11:13:23.185 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T11:13:23.313 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-12-09T11:13:23.504 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T11:13:23.657 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-12-09T11:13:23.659 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:23.851 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T11:13:23.990 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-12-09T11:13:23.992 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:24.195 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T11:13:24.334 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-12-09T11:13:24.535 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T11:13:24.662 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-12-09T11:13:24.852 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T11:13:24.991 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-12-09T11:13:25.173 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T11:13:25.331 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-12-09T11:13:25.522 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T11:13:25.658 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-12-09T11:13:25.859 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T11:13:25.984 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-12-09T11:13:26.208 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T11:13:26.334 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-12-09T11:13:26.534 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T11:13:26.661 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-12-09T11:13:26.860 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T11:13:26.947 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-12-09T11:13:27.147 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T11:13:27.288 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-12-09T11:13:27.394 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T11:13:27.625 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-12-09T11:13:27.689 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T11:13:27.960 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-12-09T11:13:27.993 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T11:13:28.307 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-12-09T11:13:28.349 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T11:13:28.617 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-12-09T11:13:28.660 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T11:13:28.950 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-12-09T11:13:28.983 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T11:13:29.296 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-12-09T11:13:29.307 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T11:13:29.646 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-12-09T11:13:29.653 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T11:13:29.929 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T11:13:29.951 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T11:13:30.172 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T11:13:30.288 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-12-09T11:13:30.548 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T11:13:30.614 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-12-09T11:13:30.865 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T11:13:30.960 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-12-09T11:13:31.148 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T11:13:31.290 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-12-09T11:13:31.479 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T11:13:31.628 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-12-09T11:13:31.820 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T11:13:31.948 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-12-09T11:13:32.170 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T11:13:32.301 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-12-09T11:13:32.500 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T11:13:32.638 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T11:13:32.824 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T11:13:32.969 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-12-09T11:13:33.149 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T11:13:33.308 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-12-09T11:13:33.498 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T11:13:33.677 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-12-09T11:13:33.813 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T11:13:33.994 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-12-09T11:13:34.164 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T11:13:34.363 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-12-09T11:13:34.510 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T11:13:34.678 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-12-09T11:13:34.834 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T11:13:35.036 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-12-09T11:13:35.197 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T11:13:35.375 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-12-09T11:13:35.522 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T11:13:35.681 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-12-09T11:13:35.850 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T11:13:36.016 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-12-09T11:13:36.173 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T11:13:36.374 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-12-09T11:13:36.492 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T11:13:36.725 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-12-09T11:13:36.820 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T11:13:37.082 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-12-09T11:13:37.146 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T11:13:37.374 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-12-09T11:13:37.482 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T11:13:37.734 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-12-09T11:13:37.829 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T11:13:38.078 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-12-09T11:13:38.184 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T11:13:38.436 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-12-09T11:13:38.523 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T11:13:38.794 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-12-09T11:13:38.860 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T11:13:39.135 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-12-09T11:13:39.200 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T11:13:39.451 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-12-09T11:13:39.525 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T11:13:39.798 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-12-09T11:13:39.851 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T11:13:40.125 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-12-09T11:13:40.158 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T11:13:40.453 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-12-09T11:13:40.486 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T11:13:40.804 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-12-09T11:13:40.805 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T11:13:41.129 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T11:13:41.173 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-12-09T11:13:41.454 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T11:13:41.518 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-12-09T11:13:41.791 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T11:13:41.865 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-12-09T11:13:42.137 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T11:13:42.203 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-12-09T11:13:42.466 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T11:13:42.550 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-12-09T11:13:42.552 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:42.795 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T11:13:42.891 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-12-09T11:13:43.147 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T11:13:43.211 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-12-09T11:13:43.492 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T11:13:43.567 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-12-09T11:13:43.831 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T11:13:43.929 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-12-09T11:13:44.151 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T11:13:44.284 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-12-09T11:13:44.506 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T11:13:44.622 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-12-09T11:13:44.853 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T11:13:44.942 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-12-09T11:13:45.204 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T11:13:45.278 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-12-09T11:13:45.530 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T11:13:45.635 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-12-09T11:13:45.881 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T11:13:45.978 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-12-09T11:13:46.219 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T11:13:46.324 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-12-09T11:13:46.559 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T11:13:46.680 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-12-09T11:13:46.891 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T11:13:47.002 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-12-09T11:13:47.220 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T11:13:47.359 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-12-09T11:13:47.540 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T11:13:47.718 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-12-09T11:13:47.880 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T11:13:48.048 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-12-09T11:13:48.235 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T11:13:48.403 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-12-09T11:13:48.559 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T11:13:48.735 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-12-09T11:13:48.891 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T11:13:49.043 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-12-09T11:13:49.225 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T11:13:49.384 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-12-09T11:13:49.550 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T11:13:49.739 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-12-09T11:13:49.854 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T11:13:50.084 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-12-09T11:13:50.192 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T11:13:50.426 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-12-09T11:13:50.511 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T11:13:50.745 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-12-09T11:13:51.079 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-12-09T11:13:51.081 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.185 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T11:13:51.307 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:51 +0000 (0:00:34.764) 0:00:46.922 ******* 2019-12-09T11:13:51.308 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.334 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.336 INFO:teuthology.task.ansible.out:TASK [users : Create all users without sudo access.] *************************** 2019-12-09T11:13:51.373 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:51 +0000 (0:00:00.066) 0:00:46.988 ******* 2019-12-09T11:13:51.375 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.423 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.424 INFO:teuthology.task.ansible.out:TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-12-09T11:13:51.427 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.428 INFO:teuthology.task.ansible.out:TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T11:13:51.432 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a 2019-12-09T11:13:51.433 INFO:teuthology.task.ansible.out:12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbe 2019-12-09T11:13:51.435 INFO:teuthology.task.ansible.out:njamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@p 2019-12-09T11:13:51.436 INFO:teuthology.task.ansible.out:udgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e30 2019-12-09T11:13:51.438 INFO:teuthology.task.ansible.out:7bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba 2019-12-09T11:13:51.439 INFO:teuthology.task.ansible.out:6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4 2019-12-09T11:13:51.441 INFO:teuthology.task.ansible.out:f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78 2019-12-09T11:13:51.442 INFO:teuthology.task.ansible.out:twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 2019-12-09T11:13:51.443 INFO:teuthology.task.ansible.out:6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f8 2019-12-09T11:13:51.445 INFO:teuthology.task.ansible.out:48846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8a 2019-12-09T11:13:51.446 INFO:teuthology.task.ansible.out:a01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalho 2019-12-09T11:13:51.448 INFO:teuthology.task.ansible.out:jr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T11:13:51.489 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:51 +0000 (0:00:00.115) 0:00:47.104 ******* 2019-12-09T11:13:51.491 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.758 INFO:teuthology.task.ansible.out: 2019-12-09T11:13:51.759 INFO:teuthology.task.ansible.out:TASK [users : Create all admin users with sudo access.] ************************ ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-12-09T11:13:52.454 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-12-09T11:13:52.554 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T11:13:52.556 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T11:13:52.592 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:52 +0000 (0:00:01.102) 0:00:48.207 ******* 2019-12-09T11:13:52.629 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:52 +0000 (0:00:00.036) 0:00:48.243 ******* 2019-12-09T11:13:52.651 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-12-09T11:13:52.689 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:52 +0000 (0:00:00.059) 0:00:48.303 ******* 2019-12-09T11:13:52.738 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-12-09T11:13:52.742 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-12-09T11:13:52.743 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7 2019-12-09T11:13:52.745 INFO:teuthology.task.ansible.out:L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32 2019-12-09T11:13:52.746 INFO:teuthology.task.ansible.out:f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b 2019-12-09T11:13:52.748 INFO:teuthology.task.ansible.out:09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e 2019-12-09T11:13:52.749 INFO:teuthology.task.ansible.out:66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa 2019-12-09T11:13:52.750 INFO:teuthology.task.ansible.out:8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f 2019-12-09T11:13:52.752 INFO:teuthology.task.ansible.out:826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": 2019-12-09T11:13:52.753 INFO:teuthology.task.ansible.out: "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharth 2019-12-09T11:13:52.755 INFO:teuthology.task.ansible.out:anup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c3659458 2019-12-09T11:13:52.756 INFO:teuthology.task.ansible.out:5e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-12-09T11:13:52.804 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:52 +0000 (0:00:00.115) 0:00:48.419 ******* 2019-12-09T11:13:52.998 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:13:53.000 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T11:13:53.343 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T11:13:53.499 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-12-09T11:13:53.499 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com -> localhost] => {"after": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "attempts": 1, "before": "2c3c6d863a854d6a1e155fc475e8273d88b91491", "changed": false, "remote_url_changed": false} 2019-12-09T11:13:53.546 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:13:53 +0000 (0:00:00.741) 0:00:49.161 ******* 2019-12-09T11:13:53.690 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:13:53.697 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T11:13:53.951 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-12-09T11:13:53.994 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T11:13:54.317 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-12-09T11:13:54.339 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T11:13:54.341 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T11:13:54.696 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} ok: [smith 2019-12-09T11:13:54.698 INFO:teuthology.task.ansible.out:i174.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-12-09T11:13:55.021 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T11:13:55.022 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T11:13:55.032 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-12-09T11:13:55.354 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T11:13:55.386 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-12-09T11:13:55.388 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-12-09T11:13:55.708 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T11:13:55.729 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:55.730 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-12-09T11:13:56.051 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T11:13:56.052 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T11:13:56.062 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:56.073 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:56.084 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:56.085 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-12-09T11:13:56.086 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-12-09T11:13:56.426 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T11:13:56.437 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-12-09T11:13:56.746 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T11:13:56.747 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T11:13:56.778 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-12-09T11:13:56.789 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:57.096 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T11:13:57.149 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-12-09T11:13:57.151 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-12-09T11:13:57.160 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:57.172 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:57.193 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:13:57.426 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T11:13:57.532 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-12-09T11:13:57.752 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T11:13:57.889 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-12-09T11:13:57.891 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-12-09T11:13:58.101 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T11:13:58.259 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-12-09T11:13:58.428 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T11:13:58.595 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-12-09T11:13:58.783 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T11:13:58.909 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-12-09T11:13:59.128 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T11:13:59.130 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T11:13:59.254 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-12-09T11:13:59.465 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T11:13:59.466 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T11:13:59.528 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-12-09T11:13:59.811 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T11:13:59.864 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-12-09T11:14:00.116 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T11:14:00.201 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-12-09T11:14:00.203 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-12-09T11:14:00.474 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T11:14:00.476 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T11:14:00.538 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-12-09T11:14:00.540 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-12-09T11:14:00.821 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T11:14:00.865 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUig 2019-12-09T11:14:00.866 INFO:teuthology.task.ansible.out:WVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-12-09T11:14:00.867 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:01.162 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T11:14:01.205 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-12-09T11:14:01.508 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T11:14:01.579 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-12-09T11:14:01.581 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-12-09T11:14:01.807 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T11:14:01.927 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-12-09T11:14:02.162 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T11:14:02.259 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-12-09T11:14:02.527 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T11:14:02.529 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T11:14:02.615 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-12-09T11:14:02.885 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T11:14:02.985 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-12-09T11:14:03.184 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T11:14:03.325 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-12-09T11:14:03.536 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T11:14:03.697 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-12-09T11:14:03.698 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-12-09T11:14:03.879 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T11:14:04.032 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-12-09T11:14:04.233 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T11:14:04.390 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-12-09T11:14:04.559 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T11:14:04.727 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-12-09T11:14:04.904 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T11:14:05.063 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-12-09T11:14:05.255 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T11:14:05.406 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-12-09T11:14:05.594 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T11:14:05.775 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-12-09T11:14:05.925 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T11:14:05.926 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T11:14:06.124 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T11:14:06.260 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T11:14:06.261 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T11:14:06.491 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-12-09T11:14:06.595 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T11:14:06.860 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-12-09T11:14:06.945 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T11:14:07.178 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-12-09T11:14:07.180 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-12-09T11:14:07.285 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T11:14:07.539 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-12-09T11:14:07.540 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-12-09T11:14:07.633 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T11:14:07.635 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T11:14:07.911 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-12-09T11:14:07.984 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T11:14:07.986 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T11:14:07.987 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-12-09T11:14:08.268 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-12-09T11:14:08.310 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T11:14:08.579 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-12-09T11:14:08.673 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T11:14:08.924 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-12-09T11:14:08.925 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T11:14:08.987 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T11:14:09.324 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-12-09T11:14:09.325 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-12-09T11:14:09.327 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T11:14:09.670 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-12-09T11:14:09.682 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T11:14:10.014 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T11:14:10.026 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-12-09T11:14:10.381 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-12-09T11:14:10.383 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T11:14:10.715 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-12-09T11:14:10.737 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T11:14:11.063 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-12-09T11:14:11.086 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T11:14:11.087 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T11:14:11.429 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-12-09T11:14:11.459 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T11:14:11.772 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T11:14:11.773 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-12-09T11:14:12.106 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-12-09T11:14:12.160 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-12-09T11:14:12.162 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T11:14:12.452 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-12-09T11:14:12.454 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-12-09T11:14:12.530 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T11:14:12.791 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-12-09T11:14:12.864 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T11:14:13.113 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-12-09T11:14:13.207 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T11:14:13.483 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1Yt 2019-12-09T11:14:13.485 INFO:teuthology.task.ansible.out:S7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", 2019-12-09T11:14:13.486 INFO:teuthology.task.ansible.out:"manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-12-09T11:14:13.527 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdP 2019-12-09T11:14:13.528 INFO:teuthology.task.ansible.out:Buld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T11:14:13.845 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-12-09T11:14:13.847 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:13.867 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6H 2019-12-09T11:14:13.868 INFO:teuthology.task.ansible.out:BvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T11:14:13.869 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:14.197 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T11:14:14.206 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-12-09T11:14:14.541 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T11:14:14.593 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-12-09T11:14:14.595 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:14.877 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T11:14:14.976 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-12-09T11:14:15.207 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T11:14:15.324 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-12-09T11:14:15.516 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T11:14:15.517 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:15.686 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-12-09T11:14:15.865 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T11:14:16.047 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-12-09T11:14:16.216 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T11:14:16.361 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-12-09T11:14:16.569 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T11:14:16.663 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-12-09T11:14:16.893 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T11:14:17.019 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-12-09T11:14:17.251 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T11:14:17.356 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-12-09T11:14:17.604 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T11:14:17.708 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-12-09T11:14:17.947 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T11:14:18.057 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-12-09T11:14:18.280 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T11:14:18.409 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-12-09T11:14:18.630 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T11:14:18.726 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-12-09T11:14:18.956 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T11:14:19.091 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-12-09T11:14:19.313 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T11:14:19.444 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-12-09T11:14:19.646 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T11:14:19.809 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-12-09T11:14:19.988 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T11:14:20.159 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-12-09T11:14:20.329 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T11:14:20.513 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-12-09T11:14:20.647 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T11:14:20.878 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-12-09T11:14:21.001 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T11:14:21.002 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-12-09T11:14:21.213 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-12-09T11:14:21.323 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T11:14:21.325 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T11:14:21.562 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-12-09T11:14:21.678 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T11:14:21.912 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-12-09T11:14:22.009 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T11:14:22.300 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-12-09T11:14:22.322 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T11:14:22.667 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-12-09T11:14:22.668 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T11:14:22.960 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T11:14:23.035 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-12-09T11:14:23.036 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-12-09T11:14:23.305 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T11:14:23.399 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz1UnJy64hXJKl1OsqRPlZ84m/fyHE8/Hqimy5s8sWcvRs2tL5oK30zFataqv1BC7r46+9t1eGUIRAuBg6QPuJnJCnPzS8yTVJ6DqSDGE2NAnpcMK5fSb+qgQ2adHFkjzZ4eKZbnDJMjYBtpUEp1Ve2ncdq5YCy6hGLIstkGq331ULBc/Lm3ZHBwy5I51S8eldkuP975otBYSPZaaN25QDncZzrW36zb3VwoGUAANEfHC1kLgTTFjx0bdsAGYCduXzyyXls8qSv9gbzcRzZZoH5B7tj8VDBoHszPnqGjGqwtWaelifYE/Svp+8fJKLlzRUM+XGNZoNcRl1/1XVhYV/ deepika@deepika", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-12-09T11:14:23.661 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T11:14:23.766 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-12-09T11:14:23.994 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T11:14:24.125 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-12-09T11:14:24.318 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T11:14:24.497 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-12-09T11:14:24.645 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T11:14:24.836 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-12-09T11:14:24.971 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T11:14:25.178 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-12-09T11:14:25.323 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-12-09T11:14:25.528 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-12-09T11:14:25.684 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T11:14:25.871 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-12-09T11:14:25.975 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T11:14:26.198 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-12-09T11:14:26.325 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T11:14:26.327 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T11:14:26.522 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-12-09T11:14:26.685 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T11:14:26.872 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-12-09T11:14:27.020 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-12-09T11:14:27.121 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:27 +0000 (0:00:33.574) 0:01:22.735 ******* 2019-12-09T11:14:27.170 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:27.171 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.172 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.183 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.194 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.205 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.216 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.217 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:27.218 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.229 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.240 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.573 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:27.574 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-12-09T11:14:27.576 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T11:14:27.577 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:27.901 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:27.902 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-12-09T11:14:27.943 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:27.945 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T11:14:28.224 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:28.225 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-12-09T11:14:28.227 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-12-09T11:14:28.299 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:28.301 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T11:14:28.568 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:28.569 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-12-09T11:14:28.652 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:28.653 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T11:14:28.663 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.685 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.688 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.698 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.719 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.720 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.731 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.741 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.752 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.764 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.765 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.776 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.787 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.798 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.809 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.830 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.831 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.842 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.853 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.875 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.892 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.894 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.931 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-12-09T11:14:28.960 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:28.961 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.963 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:28.964 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.002 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:28 +0000 (0:00:01.881) 0:01:24.617 ******* 2019-12-09T11:14:29.008 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.009 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.011 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.012 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.027 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.028 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.046 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.050 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.050 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.063 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.064 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.073 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.074 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.085 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.087 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.088 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.089 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.091 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.101 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.102 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.113 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.114 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.115 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.116 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.128 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.139 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.139 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.150 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.161 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.174 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.174 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.188 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.200 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.211 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.224 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.234 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.245 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.256 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.267 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.267 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.278 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.289 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.300 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.312 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.312 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.323 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.334 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.345 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.347 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.358 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.369 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": 2019-12-09T11:14:29.371 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2019-12-09T11:14:29.381 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.384 INFO:teuthology.task.ansible.out: skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.393 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.404 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.406 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.416 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.427 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.438 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.450 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.452 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.464 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.474 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.495 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-12-09T11:14:29.497 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.499 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.505 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.506 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:29.507 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.510 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.528 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.584 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:29 +0000 (0:00:00.582) 0:01:25.199 ******* 2019-12-09T11:14:29.623 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T11:14:29.624 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:29.669 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:29 +0000 (0:00:00.084) 0:01:25.284 ******* 2019-12-09T11:14:29.844 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-12-09T11:14:29.846 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-12-09T11:14:30.027 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi106.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T11:14:30.195 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-12-09T11:14:30.343 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi106.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T11:14:30.534 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-12-09T11:14:30.546 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.556 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.557 INFO:teuthology.task.ansible.out: skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.569 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.582 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.583 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.594 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.606 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.607 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.621 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.632 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.643 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.655 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.667 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:30.668 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T11:14:30.670 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.674 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.682 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.693 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.695 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.706 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.718 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.719 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.731 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.743 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.744 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.756 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.759 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.772 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.783 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.794 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.805 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.817 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.820 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.832 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.833 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.844 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.857 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.859 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.873 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.876 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.889 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.900 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.910 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.912 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.924 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.934 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.946 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.947 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.960 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.972 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.982 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.993 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:30.994 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:30.996 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T11:14:30.997 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.008 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.009 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.020 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.032 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.033 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.044 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.045 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.057 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.069 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.071 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.082 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.094 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.095 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.107 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.108 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.120 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.132 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.134 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@thinkpad HXRUSerxILmV0ejAD/Zt1g 29b44e741567e7a2e7b664d6533247d79851fc302051c9ba34103c63a2fcfacc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.145 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.147 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.158 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.170 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.181 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.193 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.204 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.215 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.226 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.238 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.249 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.250 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.261 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.272 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.273 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:31 +0000 (0:00:01.670) 0:01:26.955 ******* 2019-12-09T11:14:31.346 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:31.347 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T11:14:31.370 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2019-12-09T11:14:31.372 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:31.414 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:31 +0000 (0:00:00.071) 0:01:27.026 ******* 2019-12-09T11:14:31.611 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:31.613 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T11:14:31.791 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-12-09T11:14:31.918 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T11:14:32.136 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-12-09T11:14:32.231 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T11:14:32.486 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-12-09T11:14:32.549 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T11:14:32.823 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-12-09T11:14:32.875 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T11:14:33.155 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-12-09T11:14:33.210 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T11:14:33.527 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-12-09T11:14:33.559 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T11:14:33.859 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-12-09T11:14:33.903 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T11:14:34.188 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-12-09T11:14:34.238 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T11:14:34.513 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-12-09T11:14:34.555 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T11:14:34.842 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-12-09T11:14:34.887 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T11:14:35.175 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-12-09T11:14:35.207 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T11:14:35.500 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-12-09T11:14:35.515 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T11:14:35.828 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T11:14:35.839 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-12-09T11:14:36.146 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T11:14:36.192 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-12-09T11:14:36.464 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T11:14:36.548 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-12-09T11:14:36.861 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-12-09T11:14:37.073 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T11:14:37.211 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-12-09T11:14:37.542 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-12-09T11:14:37.650 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T11:14:37.864 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-12-09T11:14:38.163 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-12-09T11:14:38.207 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T11:14:38.500 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-12-09T11:14:38.708 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T11:14:39.077 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jj", "name": "jj", "remove": false, "state": "absent"} 2019-12-09T11:14:39.305 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T11:14:39.695 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "amarangone", "name": "amarangone", "remove": false, "state": "absent"} 2019-12-09T11:14:39.852 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T11:14:40.333 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "oprypin", "name": "oprypin", "remove": false, "state": "absent"} 2019-12-09T11:14:40.428 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T11:14:40.919 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "adamyanova", "name": "adamyanova", "remove": false, "state": "absent"} 2019-12-09T11:14:40.982 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T11:14:41.438 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sbillah", "name": "sbillah", "remove": false, "state": "absent"} 2019-12-09T11:14:41.554 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T11:14:41.991 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "onyb", "name": "onyb", "remove": false, "state": "absent"} 2019-12-09T11:14:42.104 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T11:14:42.607 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "jwilliamson", "name": "jwilliamson", "remove": false, "state": "absent"} 2019-12-09T11:14:42.670 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T11:14:43.188 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T11:14:43.283 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "kmroz", "name": "kmroz", "remove": false, "state": "absent"} 2019-12-09T11:14:43.506 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T11:14:43.789 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T11:14:43.853 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "shehbazj", "name": "shehbazj", "remove": false, "state": "absent"} 2019-12-09T11:14:44.107 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T11:14:44.432 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T11:14:44.474 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "remove": false, "state": "absent"} 2019-12-09T11:14:44.752 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T11:14:45.054 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "asheplyakov", "name": "asheplyakov", "remove": false, "state": "absent"} 2019-12-09T11:14:45.086 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T11:14:45.392 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T11:14:45.644 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "liupan", "name": "liupan", "remove": false, "state": "absent"} 2019-12-09T11:14:45.981 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-12-09T11:14:45.994 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T11:14:46.085 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:46 +0000 (0:00:14.673) 0:01:41.700 ******* 2019-12-09T11:14:46.326 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-12-09T11:14:46.597 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-12-09T11:14:46.930 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-12-09T11:14:47.139 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T11:14:47.141 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890086.11-105600829867841/source", "state": "file", "uid": 0} 2019-12-09T11:14:47.178 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:01.091) 0:01:42.792 ******* 2019-12-09T11:14:47.204 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi106.front.sepia.ceph.com] => { "msg": "Host smithi106.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T11:14:47.242 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.065) 0:01:42.857 ******* 2019-12-09T11:14:47.248 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:47.249 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-12-09T11:14:47.274 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T11:14:47.313 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.070) 0:01:42.928 ******* 2019-12-09T11:14:47.350 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T11:14:47.351 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:47.388 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.074) 0:01:43.003 ******* 2019-12-09T11:14:47.573 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:47.575 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-12-09T11:14:47.815 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.106306", "end": "2019-12-09 11:14:47.780807", "rc": 0, "start": "2019-12-09 11:14:47.674501", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T11:14:47.854 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.465) 0:01:43.468 ******* 2019-12-09T11:14:47.881 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T11:14:47.882 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:47.921 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.067) 0:01:43.536 ******* 2019-12-09T11:14:47.926 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-12-09T11:14:47.927 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-12-09T11:14:47.954 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:47.987 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:47 +0000 (0:00:00.066) 0:01:43.602 ******* 2019-12-09T11:14:48.024 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T11:14:48.026 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.064 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.076) 0:01:43.679 ******* 2019-12-09T11:14:48.091 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-09T11:14:48.092 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.128 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.064) 0:01:43.743 ******* 2019-12-09T11:14:48.155 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T11:14:48.157 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.195 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.066) 0:01:43.810 ******* 2019-12-09T11:14:48.224 INFO:teuthology.task.ansible.out: 2019-12-09T11:14:48.225 INFO:teuthology.task.ansible.out:TASK [common : Set entitlements_path] ****************************************** 2019-12-09T11:14:48.227 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.268 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.069) 0:01:43.879 ******* 2019-12-09T11:14:48.300 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-09T11:14:48.302 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:48.303 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:48.363 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.098) 0:01:43.977 ******* 2019-12-09T11:14:48.400 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T11:14:48.401 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.438 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.075) 0:01:44.053 ******* 2019-12-09T11:14:48.488 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T11:14:48.490 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** changed: [smithi174.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "fche", "name": "fche", "remove": false, "state": "absent"} 2019-12-09T11:14:48.527 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.088) 0:01:44.141 ******* 2019-12-09T11:14:48.556 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T11:14:48.557 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:48.595 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.068) 0:01:44.210 ******* 2019-12-09T11:14:48.629 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.034) 0:01:44.244 ******* 2019-12-09T11:14:48.635 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T11:14:48.637 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.675 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.045) 0:01:44.289 ******* 2019-12-09T11:14:48.721 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T11:14:48.722 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:48.758 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.084) 0:01:44.373 ******* 2019-12-09T11:14:48.812 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T11:14:48.817 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.852 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.093) 0:01:44.467 ******* 2019-12-09T11:14:48.879 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T11:14:48.880 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.067) 0:01:44.534 ******* 2019-12-09T11:14:48.948 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:48.984 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:48 +0000 (0:00:00.064) 0:01:44.599 ******* 2019-12-09T11:14:49.011 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T11:14:49.013 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.048 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.064) 0:01:44.663 ******* 2019-12-09T11:14:49.075 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:49.114 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.065) 0:01:44.729 ******* 2019-12-09T11:14:49.144 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T11:14:49.146 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.184 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.069) 0:01:44.799 ******* 2019-12-09T11:14:49.217 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T11:14:49.219 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.071) 0:01:44.871 ******* 2019-12-09T11:14:49.293 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-09T11:14:49.295 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.333 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.076) 0:01:44.948 ******* 2019-12-09T11:14:49.360 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-09T11:14:49.362 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.401 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.068) 0:01:45.016 ******* 2019-12-09T11:14:49.428 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T11:14:49.430 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.467 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.065) 0:01:45.082 ******* 2019-12-09T11:14:49.493 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T11:14:49.495 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.539 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.071) 0:01:45.154 ******* 2019-12-09T11:14:49.571 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T11:14:49.571 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:49.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.074) 0:01:45.228 ******* 2019-12-09T11:14:49.642 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T11:14:49.689 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.071) 0:01:45.300 ******* 2019-12-09T11:14:49.712 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T11:14:49.713 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:49.753 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.067) 0:01:45.368 ******* 2019-12-09T11:14:49.760 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-12-09T11:14:49.761 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "checksum": "8a303ebcf60c08d57f11c57c9682a02d84685d0b", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "c2805c19b557daad4717860e2c7029bc", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890088.65-47156269434112/source", "state": "file", "uid": 0} 2019-12-09T11:14:49.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.049) 0:01:45.417 ******* 2019-12-09T11:14:49.829 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-12-09T11:14:49.830 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => { "msg": "Host smithi174.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-12-09T11:14:49.881 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.079) 0:01:45.496 ******* 2019-12-09T11:14:49.921 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-12-09T11:14:49.924 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-12-09T11:14:49.967 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:49 +0000 (0:00:00.086) 0:01:45.582 ******* 2019-12-09T11:14:49.995 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-12-09T11:14:49.996 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.043 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.075) 0:01:45.658 ******* 2019-12-09T11:14:50.094 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T11:14:50.095 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:14:50.132 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.089) 0:01:45.747 ******* 2019-12-09T11:14:50.493 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-12-09T11:14:50.494 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.091214", "end": "2019-12-09 11:14:50.451550", "rc": 0, "start": "2019-12-09 11:14:50.360336", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-12-09T11:14:50.536 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.403) 0:01:46.151 ******* 2019-12-09T11:14:50.573 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-12-09T11:14:50.575 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.618 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.081) 0:01:46.233 ******* 2019-12-09T11:14:50.644 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-12-09T11:14:50.646 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.681 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.062) 0:01:46.295 ******* 2019-12-09T11:14:50.719 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-12-09T11:14:50.721 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.763 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.082) 0:01:46.378 ******* 2019-12-09T11:14:50.790 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-12-09T11:14:50.791 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.827 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.064) 0:01:46.442 ******* 2019-12-09T11:14:50.870 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-12-09T11:14:50.872 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.914 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.086) 0:01:46.529 ******* 2019-12-09T11:14:50.952 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-12-09T11:14:50.953 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:50.989 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:50 +0000 (0:00:00.074) 0:01:46.603 ******* 2019-12-09T11:14:51.025 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-12-09T11:14:51.026 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:51.064 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.075) 0:01:46.678 ******* 2019-12-09T11:14:51.090 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-12-09T11:14:51.091 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.140 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.076) 0:01:46.755 ******* 2019-12-09T11:14:51.166 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-12-09T11:14:51.170 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.210 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.070) 0:01:46.825 ******* 2019-12-09T11:14:51.237 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-12-09T11:14:51.239 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:51.276 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.065) 0:01:46.891 ******* 2019-12-09T11:14:51.313 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-12-09T11:14:51.315 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.357 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.081) 0:01:46.972 ******* 2019-12-09T11:14:51.384 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T11:14:51.384 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T11:14:51.386 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-12-09T11:14:51.387 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:51.429 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.071) 0:01:47.043 ******* 2019-12-09T11:14:51.459 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-12-09T11:14:51.460 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.505 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.076) 0:01:47.120 ******* 2019-12-09T11:14:51.533 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-12-09T11:14:51.534 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.572 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.067) 0:01:47.187 ******* 2019-12-09T11:14:51.613 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-12-09T11:14:51.614 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.651 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.079) 0:01:47.266 ******* 2019-12-09T11:14:51.677 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-12-09T11:14:51.679 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.715 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.063) 0:01:47.330 ******* 2019-12-09T11:14:51.741 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-12-09T11:14:51.743 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:51.779 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.064) 0:01:47.394 ******* 2019-12-09T11:14:51.805 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-12-09T11:14:51.807 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.843 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.063) 0:01:47.458 ******* 2019-12-09T11:14:51.869 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-12-09T11:14:51.871 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.063) 0:01:47.521 ******* 2019-12-09T11:14:51.943 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-12-09T11:14:51.945 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:51.980 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:51 +0000 (0:00:00.073) 0:01:47.595 ******* 2019-12-09T11:14:52.006 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-12-09T11:14:52.008 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:52.044 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.064) 0:01:47.659 ******* 2019-12-09T11:14:52.071 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-12-09T11:14:52.073 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:52.108 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.063) 0:01:47.723 ******* 2019-12-09T11:14:52.144 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-12-09T11:14:52.146 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:52.182 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.074) 0:01:47.797 ******* 2019-12-09T11:14:52.219 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-12-09T11:14:52.221 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-12-09T11:14:52.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.074) 0:01:47.871 ******* 2019-12-09T11:14:52.268 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T11:14:52.305 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.048) 0:01:47.920 ******* 2019-12-09T11:14:52.311 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-12-09T11:14:52.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.044) 0:01:47.965 ******* 2019-12-09T11:14:52.376 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-12-09T11:14:52.378 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:14:52.413 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.063) 0:01:48.028 ******* 2019-12-09T11:14:52.745 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-12-09T11:14:52.747 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:14:52.786 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:52 +0000 (0:00:00.373) 0:01:48.401 ******* 2019-12-09T11:14:53.380 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T11:14:53.381 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890092.35-62137935157323/source", "state": "file", "uid": 0} 2019-12-09T11:14:53.814 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T11:14:53.815 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-12-09T11:14:54.601 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T11:14:54.602 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890093.39-60542154641795/source", "state": "file", "uid": 0} 2019-12-09T11:14:54.656 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:54 +0000 (0:00:01.869) 0:01:50.271 ******* 2019-12-09T11:14:54.723 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-12-09T11:14:54.724 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-12-09T11:14:54.776 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:54 +0000 (0:00:00.119) 0:01:50.391 ******* 2019-12-09T11:14:55.275 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T11:14:55.579 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T11:14:55.582 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-12-09T11:14:55.638 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:55 +0000 (0:00:00.861) 0:01:51.252 ******* 2019-12-09T11:14:55.985 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-12-09T11:14:55.987 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890094.95-233970396608958/source", "state": "file", "uid": 0} 2019-12-09T11:14:57.032 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1575890095.99-252608883614999/source", "state": "file", "uid": 0} 2019-12-09T11:14:57.174 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:57 +0000 (0:00:01.536) 0:01:52.789 ******* 2019-12-09T11:14:57.509 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-12-09T11:14:57.510 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-12-09T11:14:57.802 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T11:14:57.842 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:14:57 +0000 (0:00:00.667) 0:01:53.457 ******* 2019-12-09T11:15:07.489 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-12-09T11:15:07.491 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:11.428942", "end": "2019-12-09 11:15:07.461431", "rc": 0, "start": "2019-12-09 11:14:56.032489", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.ash.fastserv.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.ash.fastserv.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T11:15:07.533 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:07 +0000 (0:00:09.690) 0:02:03.147 ******* 2019-12-09T11:15:07.871 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T11:15:07.871 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T11:15:08.168 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T11:15:08.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.675) 0:02:03.823 ******* 2019-12-09T11:15:08.234 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T11:15:08.236 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.284 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.075) 0:02:03.898 ******* 2019-12-09T11:15:08.315 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.066) 0:02:03.965 ******* 2019-12-09T11:15:08.377 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T11:15:08.378 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.417 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.067) 0:02:04.032 ******* 2019-12-09T11:15:08.455 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T11:15:08.456 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.494 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.076) 0:02:04.109 ******* 2019-12-09T11:15:08.495 INFO:teuthology.task.ansible.out: 2019-12-09T11:15:08.534 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T11:15:08.536 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.080) 0:02:04.189 ******* 2019-12-09T11:15:08.602 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T11:15:08.603 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.066) 0:02:04.256 ******* 2019-12-09T11:15:08.679 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.719 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.076) 0:02:04.333 ******* 2019-12-09T11:15:08.745 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T11:15:08.747 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:08.785 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:08 +0000 (0:00:00.066) 0:02:04.399 ******* 2019-12-09T11:15:10.529 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T11:15:10.531 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:01.442752", "end": "2019-12-09 11:15:10.500424", "rc": 0, "start": "2019-12-09 11:15:09.057672", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:15:10.575 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:10 +0000 (0:00:01.790) 0:02:06.190 ******* 2019-12-09T11:15:11.110 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-12-09T11:15:11.112 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.939950", "end": "2019-12-09 11:15:11.075110", "rc": 0, "start": "2019-12-09 11:14:58.135160", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: mirror.ash.fastserv.com\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: mirror.ash.fastserv.com", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "Metadata Cache Created"]} 2019-12-09T11:15:11.151 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.576) 0:02:06.766 ******* 2019-12-09T11:15:11.276 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-12-09T11:15:11.278 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T11:15:11.316 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.164) 0:02:06.931 ******* 2019-12-09T11:15:11.343 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T11:15:11.345 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:11.382 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.065) 0:02:06.997 ******* 2019-12-09T11:15:11.409 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T11:15:11.410 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:11.447 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.065) 0:02:07.062 ******* 2019-12-09T11:15:11.476 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T11:15:11.478 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:11.514 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.066) 0:02:07.129 ******* 2019-12-09T11:15:11.520 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T11:15:11.521 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-12-09T11:15:11.544 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T11:15:11.546 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:11.582 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.067) 0:02:07.197 ******* 2019-12-09T11:15:11.825 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-12-09T11:15:11.826 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-12-09T11:15:11.884 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.302) 0:02:07.499 ******* 2019-12-09T11:15:11.910 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T11:15:11.912 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:11.948 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:11 +0000 (0:00:00.063) 0:02:07.563 ******* 2019-12-09T11:15:11.974 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-12-09T11:15:11.976 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.011 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.063) 0:02:07.626 ******* 2019-12-09T11:15:12.037 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-12-09T11:15:12.038 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.084 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.064) 0:02:07.691 ******* 2019-12-09T11:15:12.104 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-12-09T11:15:12.106 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.144 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.067) 0:02:07.759 ******* 2019-12-09T11:15:12.181 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-12-09T11:15:12.183 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.231 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.086) 0:02:07.846 ******* 2019-12-09T11:15:12.264 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-12-09T11:15:12.266 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.302 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.070) 0:02:07.916 ******* 2019-12-09T11:15:12.328 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-12-09T11:15:12.330 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.366 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.064) 0:02:07.981 ******* 2019-12-09T11:15:12.393 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-12-09T11:15:12.395 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:12.434 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.067) 0:02:08.049 ******* 2019-12-09T11:15:12.440 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T11:15:12.441 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T11:15:12.483 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:12 +0000 (0:00:00.048) 0:02:08.098 ******* 2019-12-09T11:15:13.528 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-12-09T11:15:13.530 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.767455", "end": "2019-12-09 11:15:13.500480", "rc": 0, "start": "2019-12-09 11:15:12.733025", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:15:13.576 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:13 +0000 (0:00:01.092) 0:02:09.190 ******* 2019-12-09T11:15:14.088 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-12-09T11:15:14.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:14 +0000 (0:00:00.550) 0:02:09.741 ******* 2019-12-09T11:15:14.152 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-12-09T11:15:14.154 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:14.189 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:14 +0000 (0:00:00.063) 0:02:09.804 ******* 2019-12-09T11:15:14.217 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-12-09T11:15:14.218 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:14.257 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:14 +0000 (0:00:00.067) 0:02:09.872 ******* 2019-12-09T11:15:14.283 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-12-09T11:15:14.285 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:14.324 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:14 +0000 (0:00:00.067) 0:02:09.939 ******* 2019-12-09T11:15:14.350 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-12-09T11:15:14.351 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:14.388 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:14 +0000 (0:00:00.063) 0:02:10.003 ******* 2019-12-09T11:15:15.310 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-12-09T11:15:15.311 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-12-09T11:15:15.361 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:15 +0000 (0:00:00.972) 0:02:10.976 ******* 2019-12-09T11:15:16.266 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T11:15:16.267 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T11:15:16.310 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:16 +0000 (0:00:00.948) 0:02:11.925 ******* 2019-12-09T11:15:17.204 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T11:15:17.206 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T11:15:18.043 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T11:15:18.808 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T11:15:18.857 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:18 +0000 (0:00:02.546) 0:02:14.472 ******* 2019-12-09T11:15:19.174 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-09T11:15:19.175 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:15:19.211 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:19 +0000 (0:00:00.353) 0:02:14.825 ******* 2019-12-09T11:15:19.843 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-12-09T11:15:19.844 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-12-09T11:15:19.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:19 +0000 (0:00:00.671) 0:02:15.497 ******* 2019-12-09T11:15:20.060 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T11:15:20.061 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T11:15:20.826 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T11:15:20.828 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-12-09T11:15:20.879 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T11:15:20.881 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T11:15:21.700 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T11:15:21.739 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:21 +0000 (0:00:01.856) 0:02:17.354 ******* 2019-12-09T11:15:21.755 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T11:15:21.756 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-12-09T11:15:21.766 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T11:15:21.768 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:21.805 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:21 +0000 (0:00:00.065) 0:02:17.419 ******* 2019-12-09T11:15:21.842 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T11:15:21.843 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T11:15:21.878 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:21 +0000 (0:00:00.073) 0:02:17.493 ******* 2019-12-09T11:15:22.185 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* changed: [smithi106.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T11:15:22.226 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:22 +0000 (0:00:00.347) 0:02:17.841 ******* 2019-12-09T11:15:22.269 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T11:15:22.271 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:22.281 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:22.303 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:22.314 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:22.355 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:22 +0000 (0:00:00.129) 0:02:17.970 ******* 2019-12-09T11:15:22.625 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2019-12-09T11:15:22.626 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-12-09T11:15:22.665 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:22 +0000 (0:00:00.310) 0:02:18.280 ******* 2019-12-09T11:15:22.985 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-12-09T11:15:22.988 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:15:23.024 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:23 +0000 (0:00:00.358) 0:02:18.639 ******* 2019-12-09T11:15:23.900 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-12-09T11:15:23.901 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-12-09T11:15:24.741 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-12-09T11:15:25.603 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-12-09T11:15:25.643 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:25 +0000 (0:00:02.618) 0:02:21.258 ******* 2019-12-09T11:15:25.669 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-12-09T11:15:25.670 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:25.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:25 +0000 (0:00:00.066) 0:02:21.324 ******* 2019-12-09T11:15:25.726 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T11:15:25.726 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T11:15:25.772 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:25 +0000 (0:00:00.062) 0:02:21.387 ******* 2019-12-09T11:15:25.778 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-12-09T11:15:25.779 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-12-09T11:15:25.820 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:25 +0000 (0:00:00.048) 0:02:21.435 ******* 2019-12-09T11:15:25.827 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T11:15:25.828 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:25.867 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:25 +0000 (0:00:00.046) 0:02:21.482 ******* 2019-12-09T11:15:26.149 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-12-09T11:15:26.151 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-12-09T11:15:26.188 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:26 +0000 (0:00:00.321) 0:02:21.803 ******* 2019-12-09T11:15:26.246 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-12-09T11:15:26.248 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:26.258 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:26.270 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:26.292 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-12-09T11:15:26.293 INFO:teuthology.task.ansible.out: 2019-12-09T11:15:26.341 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:26 +0000 (0:00:00.152) 0:02:21.955 ******* 2019-12-09T11:15:26.819 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T11:15:26.820 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T11:15:26.884 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:26 +0000 (0:00:00.542) 0:02:22.498 ******* 2019-12-09T11:15:27.760 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T11:15:27.761 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T11:15:27.798 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:27 +0000 (0:00:00.914) 0:02:23.413 ******* 2019-12-09T11:15:27.827 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T11:15:27.829 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:27.866 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:27 +0000 (0:00:00.068) 0:02:23.481 ******* 2019-12-09T11:15:28.173 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T11:15:28.175 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:15:28.213 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:28 +0000 (0:00:00.346) 0:02:23.827 ******* 2019-12-09T11:15:28.585 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T11:15:28.643 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:28 +0000 (0:00:00.430) 0:02:24.258 ******* 2019-12-09T11:15:28.680 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T11:15:28.682 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:28.716 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:28 +0000 (0:00:00.073) 0:02:24.331 ******* 2019-12-09T11:15:29.612 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T11:15:29.613 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T11:15:29.654 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:29 +0000 (0:00:00.937) 0:02:25.269 ******* 2019-12-09T11:15:29.744 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-12-09T11:15:29.745 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-12-09T11:15:29.810 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:29 +0000 (0:00:00.155) 0:02:25.424 ******* 2019-12-09T11:15:29.838 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-12-09T11:15:29.839 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:29.882 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:29 +0000 (0:00:00.072) 0:02:25.497 ******* 2019-12-09T11:15:30.502 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 11:12:48 UTC", "ActiveEnterTimestampMonotonic": "19872170", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "var-run.mount system.slice network.target basic.target nss-lookup.target local-fs.target time-sync.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 11:12:48 UTC", "AssertTimestampMonotonic": "19871608", "Before": "xdm.service getty@tty1.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 11:12:48 UTC", "ConditionTimestampMonotonic": "19871608", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8963", "ExecMainStartTimestamp": "Mon 2019-12-09 11:12:48 UTC", "ExecMainStartTimestampMonotonic": "19872123", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbi 2019-12-09T11:15:30.503 INFO:teuthology.task.ansible.out:n/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Mon 2019-12-09 11:12:48 UTC] ; stop_time=[n/a] ; pid=8963 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 11:12:48 UTC", "InactiveExitTimestampMonotonic": "19872170", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8963", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", 2019-12-09T11:15:30.505 INFO:teuthology.task.ansible.out: "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 11:12:48 UTC", "WatchdogTimestampMonotonic": "19872153", "WatchdogUSec": "0"}} 2019-12-09T11:15:30.539 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:30 +0000 (0:00:00.657) 0:02:26.154 ******* 2019-12-09T11:15:30.825 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-12-09T11:15:30.826 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-12-09T11:15:30.864 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:30 +0000 (0:00:00.325) 0:02:26.479 ******* 2019-12-09T11:15:30.869 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T11:15:30.870 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004570", "end": "2019-12-09 11:15:30.824274", "rc": 0, "start": "2019-12-09 11:15:30.819704", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T11:15:30.910 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:30 +0000 (0:00:00.045) 0:02:26.524 ******* 2019-12-09T11:15:31.384 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T11:15:31.386 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T11:15:31.423 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:31 +0000 (0:00:00.513) 0:02:27.038 ******* 2019-12-09T11:15:31.723 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-12-09T11:15:31.724 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-12-09T11:15:31.762 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:31 +0000 (0:00:00.338) 0:02:27.377 ******* 2019-12-09T11:15:31.788 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-12-09T11:15:31.790 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:31.840 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:31 +0000 (0:00:00.075) 0:02:27.452 ******* 2019-12-09T11:15:32.165 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-12-09T11:15:32.166 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:15:32.205 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:32 +0000 (0:00:00.368) 0:02:27.820 ******* 2019-12-09T11:15:32.573 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-12-09T11:15:32.575 INFO:teuthology.task.ansible.out:fatal: [smithi174.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-12-09T11:15:32.609 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:32 +0000 (0:00:00.403) 0:02:28.224 ******* 2019-12-09T11:15:32.635 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-12-09T11:15:32.636 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:15:32.673 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:32 +0000 (0:00:00.063) 0:02:28.288 ******* 2019-12-09T11:15:33.591 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-12-09T11:15:33.592 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-12-09T11:15:33.631 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:33 +0000 (0:00:00.957) 0:02:29.246 ******* 2019-12-09T11:15:34.101 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-12-09T11:15:34.103 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-12-09 11:12:28 UTC", "ActiveEnterTimestampMonotonic": "18985844", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "time-sync.target var-run.mount network.target basic.target systemd-journald.socket system.slice local-fs.target nss-lookup.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-12-09 11:12:28 UTC", "AssertTimestampMonotonic": "18985230", "Before": "xdm.service multi-user.target getty@tty1.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-12-09 11:12:28 UTC", "ConditionTimestampMonotonic": "18985230", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8958", "ExecMainStartTimestamp": "Mon 2019-12-09 11:12:28 UTC", "ExecMainStartTimestampMonotonic": "18985790", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-12-09T11:15:34.104 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-12-09 11:12:28 UTC] ; stop_time=[n/a] ; pid=8958 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-12-09 11:12:28 UTC", "InactiveExitTimestampMonotonic": "18985844", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8958", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-12-09T11:15:34.106 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-12-09 11:12:28 UTC", "WatchdogTimestampMonotonic": "18985822", "WatchdogUSec": "0"}} 2019-12-09T11:15:34.141 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:34 +0000 (0:00:00.510) 0:02:29.756 ******* 2019-12-09T11:15:34.466 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-12-09T11:15:34.467 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004428", "end": "2019-12-09 11:15:34.435942", "rc": 0, "start": "2019-12-09 11:15:34.431514", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-12-09T11:15:34.503 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:34 +0000 (0:00:00.361) 0:02:30.118 ******* 2019-12-09T11:15:35.003 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-12-09T11:15:35.005 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-12-09T11:15:35.048 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:35 +0000 (0:00:00.544) 0:02:30.663 ******* 2019-12-09T11:15:57.319 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n-------------------------------- 2019-12-09T11:15:57.321 INFO:teuthology.task.ansible.out:------------------------------------------------\nTotal 26 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T11:15:57.356 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:57 +0000 (0:00:22.308) 0:02:52.971 ******* 2019-12-09T11:15:58.013 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-09T11:15:58.015 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T11:15:58.052 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:15:58 +0000 (0:00:00.695) 0:02:53.667 ******* 2019-12-09T11:16:02.639 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-12-09T11:16:02.640 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "changes": {"installed": [], "updated": [["selinux-policy-targeted", "3.13.1-252.el7_7.6.noarch from updates"]]}, "msg": "", "rc": 0, "results": ["Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Processing Dependency: selinux-policy = 3.13.1-252.el7_7.6 for package: selinux-policy-targeted-3.13.1-252.el7_7.6.noarch\n--> Running transaction check\n---> Package selinux-policy.noarch 0:3.13.1-252.el7.1 will be updated\n---> Package selinux-policy.noarch 0:3.13.1-252.el7_7.6 will be an update\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nUpdating:\n selinux-policy-targeted noarch 3.13.1-252.el7_7.6 updates 7.0 M\nUpdating for dependencies:\n selinux-policy noarch 3.13.1-252.el7_7.6 updates 492 k\n\nTransaction Summary\n================================================================================\nUpgrade 1 Package (+1 Dependent package)\n\nTotal download size: 7.5 M\nDownloading packages:\nDelta RPMs disabled because /usr/bin/applydeltarpm not installed.\n--------------------------------------------------------------------------------\nTotal 2019-12-09T11:16:02.642 INFO:teuthology.task.ansible.out: 26 MB/s | 7.5 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Updating : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Updating : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Cleanup : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Cleanup : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n Verifying : selinux-policy-3.13.1-252.el7_7.6.noarch 1/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7_7.6.noarch 2/4 \n Verifying : selinux-policy-targeted-3.13.1-252.el7.1.noarch 3/4 \n Verifying : selinux-policy-3.13.1-252.el7.1.noarch 4/4 \n\nUpdated:\n selinux-policy-targeted.noarch 0:3.13.1-252.el7_7.6 \n\nDependency Updated:\n selinux-policy.noarch 0:3.13.1-252.el7_7.6 \n\nComplete!\n"]} 2019-12-09T11:16:02.683 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:02 +0000 (0:00:04.625) 0:02:58.293 ******* 2019-12-09T11:16:03.218 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-12-09T11:16:03.220 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-12-09T11:16:03.256 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:03 +0000 (0:00:00.577) 0:02:58.871 ******* 2019-12-09T11:16:06.111 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-09T11:16:06.113 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.751250", "end": "2019-12-09 11:16:06.083651", "failed_when_result": false, "rc": 0, "start": "2019-12-09 11:15:58.332401", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T11:16:06.149 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:06 +0000 (0:00:02.892) 0:03:01.764 ******* 2019-12-09T11:16:07.035 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2019-12-09T11:16:07.036 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T11:16:07.073 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:07 +0000 (0:00:00.924) 0:03:02.688 ******* 2019-12-09T11:16:07.408 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T11:16:07.409 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.011790", "end": "2019-12-09 11:16:07.376884", "rc": 0, "start": "2019-12-09 11:16:07.365094", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T11:16:07.446 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:07 +0000 (0:00:00.373) 0:03:03.061 ******* 2019-12-09T11:16:07.770 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T11:16:07.771 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.023329", "end": "2019-12-09 11:16:07.744441", "rc": 0, "start": "2019-12-09 11:16:07.721112", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:07.808 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:07 +0000 (0:00:00.361) 0:03:03.423 ******* 2019-12-09T11:16:11.633 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-12-09T11:16:11.634 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.075819", "end": "2019-12-09 11:16:11.601278", "failed_when_result": false, "rc": 0, "start": "2019-12-09 11:16:03.525459", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-12-09T11:16:11.673 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:11 +0000 (0:00:03.865) 0:03:07.288 ******* 2019-12-09T11:16:12.607 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-12-09T11:16:12.641 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:12 +0000 (0:00:00.968) 0:03:08.256 ******* 2019-12-09T11:16:12.970 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-12-09T11:16:12.971 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.021437", "end": "2019-12-09 11:16:12.939189", "rc": 0, "start": "2019-12-09 11:16:12.917752", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-12-09T11:16:13.010 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:13 +0000 (0:00:00.368) 0:03:08.625 ******* 2019-12-09T11:16:13.011 INFO:teuthology.task.ansible.out: 2019-12-09T11:16:13.351 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-12-09T11:16:13.353 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.013237", "end": "2019-12-09 11:16:13.322308", "rc": 0, "start": "2019-12-09 11:16:13.309071", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:13.398 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:13 +0000 (0:00:00.388) 0:03:09.013 ******* 2019-12-09T11:16:15.961 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-09T11:16:15.962 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.832469", "end": "2019-12-09 11:16:15.925891", "rc": 0, "start": "2019-12-09 11:16:08.093422", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:15.997 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:15 +0000 (0:00:02.599) 0:03:11.612 ******* 2019-12-09T11:16:16.336 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T11:16:16.338 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T11:16:16.374 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.376) 0:03:11.988 ******* 2019-12-09T11:16:16.400 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T11:16:16.401 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T11:16:16.438 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.063) 0:03:12.052 ******* 2019-12-09T11:16:16.474 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T11:16:16.475 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T11:16:16.511 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.073) 0:03:12.126 ******* 2019-12-09T11:16:16.548 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T11:16:16.549 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T11:16:16.584 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.072) 0:03:12.199 ******* 2019-12-09T11:16:16.621 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T11:16:16.623 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T11:16:16.657 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.072) 0:03:12.272 ******* 2019-12-09T11:16:16.961 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T11:16:16.962 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T11:16:16.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:16 +0000 (0:00:00.338) 0:03:12.610 ******* 2019-12-09T11:16:17.299 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T11:16:17.301 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T11:16:17.335 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:17 +0000 (0:00:00.339) 0:03:12.950 ******* 2019-12-09T11:16:17.361 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T11:16:17.363 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:17.396 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:17 +0000 (0:00:00.060) 0:03:13.011 ******* 2019-12-09T11:16:17.423 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T11:16:17.424 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:17.459 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:17 +0000 (0:00:00.062) 0:03:13.074 ******* 2019-12-09T11:16:17.807 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T11:16:17.808 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T11:16:17.843 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:17 +0000 (0:00:00.383) 0:03:13.457 ******* 2019-12-09T11:16:18.841 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T11:16:18.843 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T11:16:18.883 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:18 +0000 (0:00:01.040) 0:03:14.498 ******* 2019-12-09T11:16:18.921 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T11:16:18.922 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi106"}, "changed": false} 2019-12-09T11:16:18.957 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:18 +0000 (0:00:00.073) 0:03:14.572 ******* 2019-12-09T11:16:19.892 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi106.front.sepia.ceph.com", "ansible_hostname": "smithi106", "ansible_nodename": "smithi106"}, "changed": true, "name": "smithi106"} 2019-12-09T11:16:19.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:19 +0000 (0:00:00.973) 0:03:15.545 ******* 2019-12-09T11:16:20.774 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T11:16:20.776 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T11:16:20.813 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:20 +0000 (0:00:00.882) 0:03:16.428 ******* 2019-12-09T11:16:21.446 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2019-12-09T11:16:21.447 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.719118", "end": "2019-12-09 11:16:21.414123", "rc": 0, "start": "2019-12-09 11:16:13.695005", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:21.487 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:21 +0000 (0:00:00.674) 0:03:17.102 ******* 2019-12-09T11:16:21.675 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T11:16:21.676 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T11:16:21.713 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:21 +0000 (0:00:00.225) 0:03:17.328 ******* 2019-12-09T11:16:21.843 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2019-12-09T11:16:21.845 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-12-09T11:16:21.893 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:21 +0000 (0:00:00.178) 0:03:17.507 ******* 2019-12-09T11:16:21.921 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2019-12-09T11:16:21.922 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-12-09T11:16:21.958 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:21 +0000 (0:00:00.065) 0:03:17.573 ******* 2019-12-09T11:16:21.995 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-12-09T11:16:21.996 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T11:16:22.035 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.077) 0:03:17.650 ******* 2019-12-09T11:16:22.074 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-12-09T11:16:22.075 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-12-09T11:16:22.077 INFO:teuthology.task.ansible.out: 2019-12-09T11:16:22.115 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.079) 0:03:17.730 ******* 2019-12-09T11:16:22.153 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-12-09T11:16:22.154 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-12-09T11:16:22.193 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.078) 0:03:17.808 ******* 2019-12-09T11:16:22.509 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-12-09T11:16:22.510 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-12-09T11:16:22.546 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.353) 0:03:18.161 ******* 2019-12-09T11:16:22.594 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T11:16:22.595 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T11:16:22.631 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.085) 0:03:18.246 ******* 2019-12-09T11:16:22.856 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-12-09T11:16:22.857 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-12-09T11:16:22.893 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.262) 0:03:18.508 ******* 2019-12-09T11:16:22.920 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-12-09T11:16:22.921 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:22.957 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:22 +0000 (0:00:00.063) 0:03:18.572 ******* 2019-12-09T11:16:22.983 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-12-09T11:16:22.984 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:23.019 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:23 +0000 (0:00:00.062) 0:03:18.634 ******* 2019-12-09T11:16:23.377 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-12-09T11:16:23.378 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-12-09T11:16:23.415 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:23 +0000 (0:00:00.395) 0:03:19.030 ******* 2019-12-09T11:16:23.556 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T11:16:23.557 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T11:16:23.594 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:23 +0000 (0:00:00.178) 0:03:19.209 ******* 2019-12-09T11:16:23.922 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T11:16:23.923 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T11:16:23.960 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:23 +0000 (0:00:00.366) 0:03:19.575 ******* 2019-12-09T11:16:24.288 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T11:16:24.289 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T11:16:24.326 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:24 +0000 (0:00:00.366) 0:03:19.941 ******* 2019-12-09T11:16:24.394 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-12-09T11:16:24.394 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-12-09T11:16:24.430 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:24 +0000 (0:00:00.104) 0:03:20.045 ******* 2019-12-09T11:16:24.456 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-12-09T11:16:24.458 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi174"}, "changed": false} 2019-12-09T11:16:24.493 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:24 +0000 (0:00:00.062) 0:03:20.108 ******* 2019-12-09T11:16:24.677 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T11:16:24.678 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T11:16:24.715 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:24 +0000 (0:00:00.222) 0:03:20.330 ******* 2019-12-09T11:16:25.053 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2019-12-09T11:16:25.055 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi174.front.sepia.ceph.com", "ansible_hostname": "smithi174", "ansible_nodename": "smithi174"}, "changed": true, "name": "smithi174"} 2019-12-09T11:16:25.097 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:25 +0000 (0:00:00.381) 0:03:20.712 ******* 2019-12-09T11:16:25.672 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T11:16:25.673 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T11:16:25.708 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:25 +0000 (0:00:00.611) 0:03:21.323 ******* 2019-12-09T11:16:25.963 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-12-09T11:16:25.964 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-12-09T11:16:25.999 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:25 +0000 (0:00:00.290) 0:03:21.614 ******* 2019-12-09T11:16:26.635 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T11:16:26.636 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T11:16:26.673 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:26 +0000 (0:00:00.673) 0:03:22.288 ******* 2019-12-09T11:16:26.929 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-12-09T11:16:26.930 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-12-09T11:16:26.965 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:26 +0000 (0:00:00.291) 0:03:22.579 ******* 2019-12-09T11:16:27.836 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-12-09T11:16:27.838 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-12-09T11:16:27.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:27 +0000 (0:00:00.908) 0:03:23.488 ******* 2019-12-09T11:16:28.780 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-12-09T11:16:28.780 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-12-09T11:16:28.818 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:28 +0000 (0:00:00.944) 0:03:24.433 ******* 2019-12-09T11:16:29.082 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T11:16:29.084 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.088109", "end": "2019-12-09 11:16:29.053591", "rc": 0, "start": "2019-12-09 11:16:26.965482", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:29.123 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:29 +0000 (0:00:00.305) 0:03:24.738 ******* 2019-12-09T11:16:29.150 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T11:16:29.151 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T11:16:29.152 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T11:16:29.190 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:29 +0000 (0:00:00.067) 0:03:24.805 ******* 2019-12-09T11:16:29.225 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:29 +0000 (0:00:00.034) 0:03:24.840 ******* 2019-12-09T11:16:29.535 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T11:16:29.536 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-12-09T11:16:29.575 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:29 +0000 (0:00:00.347) 0:03:25.187 ******* 2019-12-09T11:16:29.919 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T11:16:29.919 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-12-09T11:16:29.957 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:29 +0000 (0:00:00.384) 0:03:25.572 ******* 2019-12-09T11:16:30.108 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T11:16:30.109 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T11:16:30.851 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T11:16:30.852 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-12-09T11:16:30.889 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:30 +0000 (0:00:00.932) 0:03:26.504 ******* 2019-12-09T11:16:31.033 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T11:16:31.035 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T11:16:31.075 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.185) 0:03:26.690 ******* 2019-12-09T11:16:31.112 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T11:16:31.114 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:31.151 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.075) 0:03:26.766 ******* 2019-12-09T11:16:31.187 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T11:16:31.188 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:16:31.199 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:16:31.238 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.086) 0:03:26.853 ******* 2019-12-09T11:16:31.275 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T11:16:31.276 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:31.315 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.076) 0:03:26.929 ******* 2019-12-09T11:16:31.352 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:16:31.354 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T11:16:31.391 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.075) 0:03:27.005 ******* 2019-12-09T11:16:31.810 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T11:16:31.811 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-12-09T11:16:31.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.459) 0:03:27.465 ******* 2019-12-09T11:16:31.898 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T11:16:31.899 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T11:16:31.943 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:31 +0000 (0:00:00.092) 0:03:27.558 ******* 2019-12-09T11:16:32.621 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T11:16:32.622 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.341997", "end": "2019-12-09 11:16:32.587650", "rc": 0, "start": "2019-12-09 11:16:32.245653", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.ash.fastserv.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T11:16:32.660 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:32 +0000 (0:00:00.716) 0:03:28.275 ******* 2019-12-09T11:16:33.019 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T11:16:33.020 INFO:teuthology.task.ansible.out:fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.047968", "end": "2019-12-09 11:16:32.985640", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 11:16:32.937672", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T11:16:33.058 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:33 +0000 (0:00:00.397) 0:03:28.673 ******* 2019-12-09T11:16:33.085 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T11:16:33.087 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:33.123 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:33 +0000 (0:00:00.064) 0:03:28.738 ******* 2019-12-09T11:16:34.253 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T11:16:34.254 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.070077", "end": "2019-12-09 11:16:34.219884", "rc": 0, "start": "2019-12-09 11:16:32.149807", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:16:34.293 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:34 +0000 (0:00:01.170) 0:03:29.908 ******* 2019-12-09T11:16:34.320 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-12-09T11:16:34.359 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:34 +0000 (0:00:00.064) 0:03:29.973 ******* 2019-12-09T11:16:35.301 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-12-09T11:16:35.302 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-12-09T11:16:36.188 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-12-09T11:16:36.229 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:36 +0000 (0:00:01.871) 0:03:31.844 ******* 2019-12-09T11:16:36.255 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-12-09T11:16:36.256 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:36.292 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:36 +0000 (0:00:00.062) 0:03:31.906 ******* 2019-12-09T11:16:36.338 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-12-09T11:16:36.340 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:16:36.350 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:16:36.388 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:36 +0000 (0:00:00.096) 0:03:32.003 ******* 2019-12-09T11:16:36.415 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-12-09T11:16:36.417 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:36.454 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:36 +0000 (0:00:00.065) 0:03:32.069 ******* 2019-12-09T11:16:36.490 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:16:36.492 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-12-09T11:16:36.534 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:36 +0000 (0:00:00.080) 0:03:32.149 ******* 2019-12-09T11:16:37.028 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-12-09T11:16:37.029 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-12-09T11:16:37.068 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:37 +0000 (0:00:00.533) 0:03:32.683 ******* 2019-12-09T11:16:37.749 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-12-09T11:16:37.749 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.330178", "end": "2019-12-09 11:16:37.713636", "rc": 0, "start": "2019-12-09 11:16:37.383458", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: mirror.ash.fastserv.com", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirrors.tripadvisor.com", " * updates: mirror.cogentco.com", "No unfinished transactions left."]} 2019-12-09T11:16:37.788 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:37 +0000 (0:00:00.720) 0:03:33.403 ******* 2019-12-09T11:16:38.156 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-12-09T11:16:38.157 INFO:teuthology.task.ansible.out:fatal: [smithi174.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.034349", "end": "2019-12-09 11:16:38.129948", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 11:16:38.095599", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-12-09T11:16:38.193 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:38 +0000 (0:00:00.404) 0:03:33.808 ******* 2019-12-09T11:16:38.228 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2019-12-09T11:16:38.230 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:16:38.266 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:38 +0000 (0:00:00.073) 0:03:33.881 ******* 2019-12-09T11:16:41.411 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T11:16:41.412 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T11:16:41.413 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T11:16:41.415 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T11:16:41.450 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:41 +0000 (0:00:03.183) 0:03:37.065 ******* 2019-12-09T11:16:44.782 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T11:16:44.783 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T11:16:44.820 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:44 +0000 (0:00:03.370) 0:03:40.435 ******* 2019-12-09T11:16:46.847 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T11:16:46.849 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-12-09T11:16:46.850 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-12-09T11:16:46.852 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T11:16:46.888 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:46 +0000 (0:00:02.067) 0:03:42.503 ******* 2019-12-09T11:16:50.588 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T11:16:50.589 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-12-09T11:16:50.628 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:16:50 +0000 (0:00:03.739) 0:03:46.242 ******* 2019-12-09T11:17:03.115 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:03.116 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T11:17:03.118 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T11:17:03.119 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies 2019-12-09T11:17:03.121 INFO:teuthology.task.ansible.out:Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 25 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T11:17:03.122 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.e 2019-12-09T11:17:03.124 INFO:teuthology.task.ansible.out:l7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T11:17:03.160 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:03 +0000 (0:00:12.532) 0:03:58.775 ******* 2019-12-09T11:17:03.928 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T11:17:03.928 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T11:17:03.963 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:03 +0000 (0:00:00.803) 0:03:59.578 ******* 2019-12-09T11:17:04.461 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T11:17:04.463 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T11:17:04.501 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:04 +0000 (0:00:00.537) 0:04:00.116 ******* 2019-12-09T11:17:04.540 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:04.542 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:04.579 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:04 +0000 (0:00:00.077) 0:04:00.194 ******* 2019-12-09T11:17:04.998 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T11:17:04.999 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "system.slice basic.target dbus.service polkit.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ebtables.service ipset.service iptables.service shutdown.target ip6tables.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T11:17:05.000 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T11:17:05.002 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T11:17:05.036 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:05 +0000 (0:00:00.457) 0:04:00.651 ******* 2019-12-09T11:17:05.420 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T11:17:05.421 INFO:teuthology.task.ansible.out:fatal: [smithi106.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T11:17:05.463 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:05 +0000 (0:00:00.426) 0:04:01.078 ******* 2019-12-09T11:17:06.268 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T11:17:06.270 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T11:17:06.308 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.844) 0:04:01.922 ******* 2019-12-09T11:17:06.335 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T11:17:06.337 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.373 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.065) 0:04:01.987 ******* 2019-12-09T11:17:06.399 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T11:17:06.437 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.064) 0:04:02.052 ******* 2019-12-09T11:17:06.464 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.500 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.062) 0:04:02.115 ******* 2019-12-09T11:17:06.526 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T11:17:06.528 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.563 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.063) 0:04:02.178 ******* 2019-12-09T11:17:06.599 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T11:17:06.601 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.637 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.073) 0:04:02.252 ******* 2019-12-09T11:17:06.663 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-09T11:17:06.665 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.703 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.066) 0:04:02.318 ******* 2019-12-09T11:17:06.730 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T11:17:06.766 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.062) 0:04:02.381 ******* 2019-12-09T11:17:06.793 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T11:17:06.794 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.829 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.063) 0:04:02.444 ******* 2019-12-09T11:17:06.855 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T11:17:06.856 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.891 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.061) 0:04:02.506 ******* 2019-12-09T11:17:06.918 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T11:17:06.919 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:06.959 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:06 +0000 (0:00:00.068) 0:04:02.574 ******* 2019-12-09T11:17:06.985 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:06.987 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.024 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:02.639 ******* 2019-12-09T11:17:07.051 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T11:17:07.053 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.089 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:02.704 ******* 2019-12-09T11:17:07.116 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:07.118 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.153 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:02.768 ******* 2019-12-09T11:17:07.179 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T11:17:07.181 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.217 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:02.832 ******* 2019-12-09T11:17:07.243 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T11:17:07.245 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.280 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.062) 0:04:02.895 ******* 2019-12-09T11:17:07.307 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-09T11:17:07.308 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.344 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:02.959 ******* 2019-12-09T11:17:07.370 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T11:17:07.372 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.408 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.063) 0:04:03.023 ******* 2019-12-09T11:17:07.435 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T11:17:07.437 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.484 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.075) 0:04:03.099 ******* 2019-12-09T11:17:07.511 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T11:17:07.512 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.563 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.079) 0:04:03.178 ******* 2019-12-09T11:17:07.590 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T11:17:07.592 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.630 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.067) 0:04:03.245 ******* 2019-12-09T11:17:07.657 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T11:17:07.659 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.709 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.078) 0:04:03.324 ******* 2019-12-09T11:17:07.735 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T11:17:07.737 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.774 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:03.389 ******* 2019-12-09T11:17:07.800 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T11:17:07.802 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.854 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.080) 0:04:03.469 ******* 2019-12-09T11:17:07.881 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-12-09T11:17:07.882 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.919 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.064) 0:04:03.533 ******* 2019-12-09T11:17:07.945 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T11:17:07.946 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:07.995 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:07 +0000 (0:00:00.076) 0:04:03.610 ******* 2019-12-09T11:17:08.022 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T11:17:08.024 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.060 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.064) 0:04:03.675 ******* 2019-12-09T11:17:08.085 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T11:17:08.087 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.136 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.075) 0:04:03.750 ******* 2019-12-09T11:17:08.162 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T11:17:08.164 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.200 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.064) 0:04:03.815 ******* 2019-12-09T11:17:08.227 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T11:17:08.228 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.263 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.063) 0:04:03.878 ******* 2019-12-09T11:17:08.290 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T11:17:08.291 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.327 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.064) 0:04:03.942 ******* 2019-12-09T11:17:08.353 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:08.354 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.389 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.061) 0:04:04.004 ******* 2019-12-09T11:17:08.416 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:08.417 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.452 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.063) 0:04:04.067 ******* 2019-12-09T11:17:08.478 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T11:17:08.480 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:08.515 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.061) 0:04:04.129 ******* 2019-12-09T11:17:08.855 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:08.856 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T11:17:08.912 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.397) 0:04:04.527 ******* 2019-12-09T11:17:08.932 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:08.933 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-12-09T11:17:08.935 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-12-09T11:17:08.936 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: mirror.ash.fastserv.com\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirrors.tripadvisor.com\n * updates: mirror.cogentco.com\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies 2019-12-09T11:17:08.938 INFO:teuthology.task.ansible.out:Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 24 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-12-09T11:17:08.939 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-12-09T11:17:08.974 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:08 +0000 (0:00:00.061) 0:04:04.589 ******* 2019-12-09T11:17:09.252 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:09.254 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:17:09.291 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.317) 0:04:04.906 ******* 2019-12-09T11:17:09.620 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:09.621 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T11:17:09.660 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.368) 0:04:05.275 ******* 2019-12-09T11:17:09.686 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T11:17:09.687 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:09.725 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.064) 0:04:05.339 ******* 2019-12-09T11:17:09.742 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-12-09T11:17:09.743 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-12-09T11:17:09.783 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.058) 0:04:05.398 ******* 2019-12-09T11:17:09.791 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:09.830 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.046) 0:04:05.445 ******* 2019-12-09T11:17:09.856 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T11:17:09.857 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:09.896 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.066) 0:04:05.511 ******* 2019-12-09T11:17:09.923 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:09.924 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:09.962 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:09 +0000 (0:00:00.065) 0:04:05.577 ******* 2019-12-09T11:17:09.988 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:09.990 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.030 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.067) 0:04:05.644 ******* 2019-12-09T11:17:10.057 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:10.058 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.094 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.064) 0:04:05.709 ******* 2019-12-09T11:17:10.121 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:10.123 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.161 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.066) 0:04:05.776 ******* 2019-12-09T11:17:10.187 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:10.189 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.224 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.063) 0:04:05.839 ******* 2019-12-09T11:17:10.250 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:10.252 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.288 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.064) 0:04:05.903 ******* 2019-12-09T11:17:10.317 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-09T11:17:10.319 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove packages] ********************************************** ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-12-09T11:17:10.356 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.068) 0:04:05.971 ******* 2019-12-09T11:17:10.391 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.034) 0:04:06.006 ******* 2019-12-09T11:17:10.396 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T11:17:10.397 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.439 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.048) 0:04:06.054 ******* 2019-12-09T11:17:10.445 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:10.446 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.486 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.046) 0:04:06.101 ******* 2019-12-09T11:17:10.490 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T11:17:10.491 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.531 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.044) 0:04:06.145 ******* 2019-12-09T11:17:10.557 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T11:17:10.558 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.593 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.062) 0:04:06.208 ******* 2019-12-09T11:17:10.620 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T11:17:10.621 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.658 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.065) 0:04:06.273 ******* 2019-12-09T11:17:10.685 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T11:17:10.686 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.720 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.061) 0:04:06.335 ******* 2019-12-09T11:17:10.746 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-09T11:17:10.747 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.782 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.062) 0:04:06.397 ******* 2019-12-09T11:17:10.809 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T11:17:10.811 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.844 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.061) 0:04:06.459 ******* 2019-12-09T11:17:10.870 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T11:17:10.871 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.907 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.062) 0:04:06.522 ******* 2019-12-09T11:17:10.923 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-12-09T11:17:10.925 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service basic.target system.slice dbus.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "iptables.service shutdown.target ip6tables.service ebtables.service ipset.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-12-09T11:17:10.927 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-12-09T11:17:10.929 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} TASK [testnode : Stop collectl] ************************************************ skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:10.962 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.054) 0:04:06.577 ******* 2019-12-09T11:17:10.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:10 +0000 (0:00:00.036) 0:04:06.613 ******* 2019-12-09T11:17:11.002 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T11:17:11.003 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.042 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.043) 0:04:06.656 ******* 2019-12-09T11:17:11.067 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T11:17:11.069 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.103 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.061) 0:04:06.718 ******* 2019-12-09T11:17:11.128 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T11:17:11.130 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.167 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.063) 0:04:06.782 ******* 2019-12-09T11:17:11.193 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:11.195 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T11:17:11.230 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.063) 0:04:06.845 ******* 2019-12-09T11:17:11.268 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T11:17:11.270 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.280 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T11:17:11.291 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.302 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.303 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.313 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.324 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.363 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.132) 0:04:06.978 ******* 2019-12-09T11:17:11.389 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2019-12-09T11:17:11.391 INFO:teuthology.task.ansible.out:fatal: [smithi174.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-12-09T11:17:11.428 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.065) 0:04:07.043 ******* 2019-12-09T11:17:11.497 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T11:17:11.499 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.005697", "end": "2019-12-09 11:17:11.478779", "rc": 0, "start": "2019-12-09 11:17:11.473082", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T11:17:11.542 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.113) 0:04:07.157 ******* 2019-12-09T11:17:11.580 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T11:17:11.581 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.621 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.079) 0:04:07.236 ******* 2019-12-09T11:17:11.658 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T11:17:11.660 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.698 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.076) 0:04:07.312 ******* 2019-12-09T11:17:11.724 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T11:17:11.726 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.761 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.063) 0:04:07.376 ******* 2019-12-09T11:17:11.787 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-09T11:17:11.789 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.826 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.064) 0:04:07.440 ******* 2019-12-09T11:17:11.831 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-12-09T11:17:11.832 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-12-09T11:17:11.870 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.044) 0:04:07.485 ******* 2019-12-09T11:17:11.907 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-12-09T11:17:11.908 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:11.944 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:11 +0000 (0:00:00.074) 0:04:07.559 ******* 2019-12-09T11:17:11.970 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-12-09T11:17:12.007 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.062) 0:04:07.621 ******* 2019-12-09T11:17:12.033 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-12-09T11:17:12.035 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.071 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.064) 0:04:07.686 ******* 2019-12-09T11:17:12.097 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-12-09T11:17:12.098 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.133 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.061) 0:04:07.747 ******* 2019-12-09T11:17:12.168 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-12-09T11:17:12.170 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi174.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.209 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.076) 0:04:07.824 ******* 2019-12-09T11:17:12.234 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-12-09T11:17:12.236 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.271 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.062) 0:04:07.886 ******* 2019-12-09T11:17:12.297 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-12-09T11:17:12.340 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.068) 0:04:07.955 ******* 2019-12-09T11:17:12.345 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T11:17:12.346 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T11:17:12.384 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.043) 0:04:07.999 ******* 2019-12-09T11:17:12.389 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-12-09T11:17:12.391 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.433 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.048) 0:04:08.048 ******* 2019-12-09T11:17:12.438 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T11:17:12.438 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item={u'block_used': 13040454, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284030, u'size_available': 930955386880, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284030, "block_size": 4096, "block_total": 240324484, "block_used": 13040454, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955386880, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T11:17:12.477 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.044) 0:04:08.092 ******* 2019-12-09T11:17:12.482 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-12-09T11:17:12.484 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.521 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.043) 0:04:08.136 ******* 2019-12-09T11:17:12.546 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-12-09T11:17:12.548 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.585 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.063) 0:04:08.199 ******* 2019-12-09T11:17:12.621 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:12.623 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.658 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.073) 0:04:08.273 ******* 2019-12-09T11:17:12.696 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-12-09T11:17:12.703 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.741 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.082) 0:04:08.355 ******* 2019-12-09T11:17:12.767 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:12.769 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.806 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.065) 0:04:08.421 ******* 2019-12-09T11:17:12.812 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T11:17:12.812 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.014195", "end": "2019-12-09 11:17:12.785278", "rc": 0, "start": "2019-12-09 11:17:12.771083", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:12.851 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.044) 0:04:08.466 ******* 2019-12-09T11:17:12.856 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-12-09T11:17:12.858 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.906 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.055) 0:04:08.521 ******* 2019-12-09T11:17:12.912 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-12-09T11:17:12.913 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'block_used': 13040454, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284030, u'size_available': 930955386880, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284030, "block_size": 4096, "block_total": 240324484, "block_used": 13040454, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955386880, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.954 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.047) 0:04:08.568 ******* 2019-12-09T11:17:12.959 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-12-09T11:17:12.961 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:12.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:12 +0000 (0:00:00.043) 0:04:08.612 ******* 2019-12-09T11:17:13.003 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T11:17:13.046 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.048) 0:04:08.661 ******* 2019-12-09T11:17:13.053 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-12-09T11:17:13.056 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.096 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.050) 0:04:08.711 ******* 2019-12-09T11:17:13.103 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:13.104 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QHPJ', u'wwn-0x5000c50091e1aea6'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QHPJ', u'wwn': u'0x5000c50091e1aea6', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', u'wwn-0x5000c50091e1aea6-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T11:17:13.106 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.117 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-12-09T11:17:13.118 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.169 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.073) 0:04:08.784 ******* 2019-12-09T11:17:13.196 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-12-09T11:17:13.198 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.238 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.068) 0:04:08.853 ******* 2019-12-09T11:17:13.264 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-12-09T11:17:13.265 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.301 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.062) 0:04:08.916 ******* 2019-12-09T11:17:13.328 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-12-09T11:17:13.329 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.367 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.065) 0:04:08.982 ******* 2019-12-09T11:17:13.393 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-12-09T11:17:13.395 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.431 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.063) 0:04:09.046 ******* 2019-12-09T11:17:13.457 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-12-09T11:17:13.459 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.496 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.065) 0:04:09.111 ******* 2019-12-09T11:17:13.522 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-12-09T11:17:13.524 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.561 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.064) 0:04:09.176 ******* 2019-12-09T11:17:13.588 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.625 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.064) 0:04:09.240 ******* 2019-12-09T11:17:13.650 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-12-09T11:17:13.652 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.686 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.060) 0:04:09.300 ******* 2019-12-09T11:17:13.712 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2019-12-09T11:17:13.714 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.749 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.063) 0:04:09.364 ******* 2019-12-09T11:17:13.774 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-12-09T11:17:13.776 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.809 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.060) 0:04:09.424 ******* 2019-12-09T11:17:13.835 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-12-09T11:17:13.837 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.873 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.063) 0:04:09.488 ******* 2019-12-09T11:17:13.898 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-12-09T11:17:13.900 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.934 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.061) 0:04:09.549 ******* 2019-12-09T11:17:13.959 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-12-09T11:17:13.961 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:13.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:13 +0000 (0:00:00.064) 0:04:09.613 ******* 2019-12-09T11:17:14.024 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-12-09T11:17:14.026 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:14.061 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:14 +0000 (0:00:00.063) 0:04:09.676 ******* 2019-12-09T11:17:14.087 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-12-09T11:17:14.089 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:14.126 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:14 +0000 (0:00:00.064) 0:04:09.741 ******* 2019-12-09T11:17:14.153 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-12-09T11:17:14.155 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:14.192 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:14 +0000 (0:00:00.065) 0:04:09.806 ******* 2019-12-09T11:17:14.530 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:14.531 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-12-09T11:17:14.573 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:14 +0000 (0:00:00.381) 0:04:10.188 ******* 2019-12-09T11:17:14.828 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:14.830 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-C55IDn-RYBU-RmT5-33FM-6V47-KDJV-jT5HOn', u'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', u'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.440128", "end": "2019-12-09 11:17:14.793184", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-C55IDn-RYBU-RmT5-33FM-6V47-KDJV-jT5HOn", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:13.353 2019-12-09T11:17:14.831 INFO:teuthology.task.ansible.out:056", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:14.894 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:14.895 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:17:14.931 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:14 +0000 (0:00:00.357) 0:04:10.546 ******* 2019-12-09T11:17:15.271 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:15.272 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-12-09T11:17:15.315 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.383) 0:04:10.930 ******* 2019-12-09T11:17:15.342 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T11:17:15.343 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.380 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.064) 0:04:10.995 ******* 2019-12-09T11:17:15.407 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T11:17:15.409 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.445 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.064) 0:04:11.060 ******* 2019-12-09T11:17:15.473 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.062) 0:04:11.122 ******* 2019-12-09T11:17:15.534 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:15.534 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.572 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.064) 0:04:11.187 ******* 2019-12-09T11:17:15.600 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:15.601 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.637 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.065) 0:04:11.252 ******* 2019-12-09T11:17:15.665 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:15.666 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.702 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.064) 0:04:11.317 ******* 2019-12-09T11:17:15.730 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-12-09T11:17:15.731 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.765 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.062) 0:04:11.380 ******* 2019-12-09T11:17:15.789 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-12-09T11:17:15.791 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.827 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.062) 0:04:11.442 ******* 2019-12-09T11:17:15.853 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-12-09T11:17:15.855 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.890 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.062) 0:04:11.505 ******* 2019-12-09T11:17:15.916 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-12-09T11:17:15.918 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:15.952 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:15 +0000 (0:00:00.062) 0:04:11.567 ******* 2019-12-09T11:17:15.978 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-12-09T11:17:15.979 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.015 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.062) 0:04:11.630 ******* 2019-12-09T11:17:16.041 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-12-09T11:17:16.043 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.081 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.065) 0:04:11.696 ******* 2019-12-09T11:17:16.107 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-12-09T11:17:16.109 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.144 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.062) 0:04:11.759 ******* 2019-12-09T11:17:16.169 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-12-09T11:17:16.169 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.208 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.064) 0:04:11.823 ******* 2019-12-09T11:17:16.226 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:16.227 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlsdMjLOibBTbPH7rn7tySGtUXtsrJ6r1E'], u'uuids': [u'a149bdce-5010-4789-b4bd-a0bad7a777b5']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.081106", "end": "2019-12-09 11:17:16.184643", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlsdMjLOibBTbPH7rn7tySGtUXtsrJ6r1E"], "labels": [], "masters": [], "uuids": ["a149bdce-5010-4789-b4bd-a0bad7a777b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:15.103537", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:16.238 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-12-09T11:17:16.240 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.275 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.066) 0:04:11.889 ******* 2019-12-09T11:17:16.300 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-12-09T11:17:16.302 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.339 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.064) 0:04:11.954 ******* 2019-12-09T11:17:16.365 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-12-09T11:17:16.367 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.404 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.064) 0:04:12.019 ******* 2019-12-09T11:17:16.430 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-12-09T11:17:16.432 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.469 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.064) 0:04:12.084 ******* 2019-12-09T11:17:16.496 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-12-09T11:17:16.497 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.533 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.063) 0:04:12.148 ******* 2019-12-09T11:17:16.558 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-12-09T11:17:16.560 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.597 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.063) 0:04:12.212 ******* 2019-12-09T11:17:16.623 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-12-09T11:17:16.625 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.659 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.061) 0:04:12.274 ******* 2019-12-09T11:17:16.694 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-12-09T11:17:16.697 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.737 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.077) 0:04:12.352 ******* 2019-12-09T11:17:16.762 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:16.763 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-12-09T11:17:16.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.064) 0:04:12.417 ******* 2019-12-09T11:17:16.848 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-12-09T11:17:16.849 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.859 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-12-09T11:17:16.870 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.871 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.882 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.893 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.893 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:16.936 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:16 +0000 (0:00:00.134) 0:04:12.551 ******* 2019-12-09T11:17:17.066 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-12-09T11:17:17.067 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003643", "end": "2019-12-09 11:17:17.046759", "rc": 0, "start": "2019-12-09 11:17:17.043116", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-12-09T11:17:17.125 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.188) 0:04:12.740 ******* 2019-12-09T11:17:17.161 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-12-09T11:17:17.163 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:17.203 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.077) 0:04:12.818 ******* 2019-12-09T11:17:17.239 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-12-09T11:17:17.241 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:17.287 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.084) 0:04:12.902 ******* 2019-12-09T11:17:17.313 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-12-09T11:17:17.315 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:17.350 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.062) 0:04:12.965 ******* 2019-12-09T11:17:17.376 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-12-09T11:17:17.377 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:17.413 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.062) 0:04:13.027 ******* 2019-12-09T11:17:17.575 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:17.576 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXllJNx2b4dAzmaP2lLRESv3FxqVhyrOY4R'], u'uuids': [u'eac49715-d0cb-43c1-9af2-c145bb8aac3f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.066001", "end": "2019-12-09 11:17:17.544408", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXllJNx2b4dAzmaP2lLRESv3FxqVhyrOY4R"], "labels": [], "masters": [], "uuids": ["eac49715-d0cb-43c1-9af2-c145bb8aac3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:16.478407", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:17.898 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-12-09T11:17:17.900 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-12-09T11:17:17.937 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:17 +0000 (0:00:00.524) 0:04:13.552 ******* 2019-12-09T11:17:17.984 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T11:17:17.986 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item={u'block_used': 13040434, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284050, u'size_available': 930955468800, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284050, "block_size": 4096, "block_total": 240324484, "block_used": 13040434, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955468800, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-12-09T11:17:18.023 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:18 +0000 (0:00:00.085) 0:04:13.637 ******* 2019-12-09T11:17:18.338 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-12-09T11:17:18.340 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013773", "end": "2019-12-09 11:17:18.307948", "rc": 0, "start": "2019-12-09 11:17:18.294175", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:18.379 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:18 +0000 (0:00:00.355) 0:04:13.993 ******* 2019-12-09T11:17:18.420 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi174.front.sepia.ceph.com] => (item={u'block_used': 13040434, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284050, u'size_available': 930955468800, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284050, "block_size": 4096, "block_total": 240324484, "block_used": 13040434, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955468800, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:18.453 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:18 +0000 (0:00:00.074) 0:04:14.068 ******* 2019-12-09T11:17:18.482 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-12-09T11:17:18.527 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:18 +0000 (0:00:00.073) 0:04:14.142 ******* 2019-12-09T11:17:18.563 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:18.564 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XN3Z', u'wwn-0x5000c50092940234'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XN3Z', u'wwn': u'0x5000c50092940234', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XN3Z-part1', u'wwn-0x5000c50092940234-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN3Z", "wwn-0x5000c50092940234"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN3Z-part1", "wwn-0x5000c50092940234-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XN3Z", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T11:17:18.566 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50092940234"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:18.930 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlJHBuSBPhdWcoBPTxDEP5XwPoUUbVrkX1'], u'uuids': [u'2e96e829-1580-4837-a70b-eee2347f9933']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.056484", "end": "2019-12-09 11:17:18.893940", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlJHBuSBPhdWcoBPTxDEP5XwPoUUbVrkX1"], "labels": [], "masters": [], "uuids": ["2e96e829-1580-4837-a70b-eee2347f9933"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:17.837456", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:20.308 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXl3s1coCxfeUjfLN3M6qxOTHsmsrbRa2eZ'], u'uuids': [u'f5546f31-7ee9-4dd9-894e-da07538d4e6e']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.065436", "end": "2019-12-09 11:17:20.269118", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXl3s1coCxfeUjfLN3M6qxOTHsmsrbRa2eZ"], "labels": [], "masters": [], "uuids": ["f5546f31-7ee9-4dd9-894e-da07538d4e6e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:19.203682", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:20.479 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-yUqXsk-Lvso-NkWd-RMSA-nvso-9TqS-ETx7zu', u'nvme-INTEL_SSDPEDMD400G4_PHFT62050074400BGN', u'nvme-nvme.8086-50484654363230353030373434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.587959", "end": "2019-12-09 11:17:20.442856", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-yUqXsk-Lvso-NkWd-RMSA-nvso-9TqS-ETx7zu", "nvme-INTEL_SSDPEDMD400G4_PHFT62050074400BGN", "nvme-nvme.8086-50484654363230353030373434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:18.854 2019-12-09T11:17:20.481 INFO:teuthology.task.ansible.out:897", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:21.682 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXldS8jYYhqfAjYerBRlZXHA2vQYrvjH2sX'], u'uuids': [u'e054f300-a394-433a-92ad-1241c30d9e48']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.055694", "end": "2019-12-09 11:17:21.644946", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXldS8jYYhqfAjYerBRlZXHA2vQYrvjH2sX"], "labels": [], "masters": [], "uuids": ["e054f300-a394-433a-92ad-1241c30d9e48"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:20.589252", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:21.727 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:21 +0000 (0:00:03.199) 0:04:17.342 ******* 2019-12-09T11:17:21.766 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:21.767 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QHPJ', u'wwn-0x5000c50091e1aea6'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5QHPJ', u'wwn': u'0x5000c50091e1aea6', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1', u'wwn-0x5000c50091e1aea6-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ", "wwn-0x5000c50091e1aea6"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QHPJ-part1", "wwn-0x5000c50091e1aea6-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QHPJ", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T11:17:21.769 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50091e1aea6"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:21.874 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:21.875 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZBU8VlgxGI6jhsUtYRY758oViWOUEKbwQ'], u'uuids': [u'58f35930-a2f8-42c9-a909-3d6e78ce0ac1']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.081630", "end": "2019-12-09 11:17:21.837665", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZBU8VlgxGI6jhsUtYRY758oViWOUEKbwQ"], "labels": [], "masters": [], "uuids": ["58f35930-a2f8-42c9-a909-3d6e78ce0ac1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:20.756035", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:22.170 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:22.171 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-C55IDn-RYBU-RmT5-33FM-6V47-KDJV-jT5HOn', u'nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN', u'nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.087746", "end": "2019-12-09 11:17:22.140280", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-C55IDn-RYBU-RmT5-33FM-6V47-KDJV-jT5HOn", "nvme-INTEL_SSDPEDMD400G4_PHFT6204013R400BGN", "nvme-nvme.8086-50484654363230343031335234303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:22.052534", "stderr": 2019-12-09T11:17:22.172 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0809236 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0809236 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:22.671 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlsdMjLOibBTbPH7rn7tySGtUXtsrJ6r1E'], u'uuids': [u'a149bdce-5010-4789-b4bd-a0bad7a777b5']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.209196", "end": "2019-12-09 11:17:22.637051", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlsdMjLOibBTbPH7rn7tySGtUXtsrJ6r1E"], "labels": [], "masters": [], "uuids": ["a149bdce-5010-4789-b4bd-a0bad7a777b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:22.427855", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.203155 s, 568 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.203155 s, 568 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:23.160 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXllJNx2b4dAzmaP2lLRESv3FxqVhyrOY4R'], u'uuids': [u'eac49715-d0cb-43c1-9af2-c145bb8aac3f']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.180409", "end": "2019-12-09 11:17:23.127776", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXllJNx2b4dAzmaP2lLRESv3FxqVhyrOY4R"], "labels": [], "masters": [], "uuids": ["eac49715-d0cb-43c1-9af2-c145bb8aac3f"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:22.947367", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.174555 s, 661 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.174555 s, 661 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:23.254 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:23.255 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZgr5cws1hcGemrEYZc0BUkl33ji6zD4yq'], u'uuids': [u'44af6425-900b-4501-8c72-82450a536761']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.070227", "end": "2019-12-09 11:17:23.213363", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZgr5cws1hcGemrEYZc0BUkl33ji6zD4yq"], "labels": [], "masters": [], "uuids": ["44af6425-900b-4501-8c72-82450a536761"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:22.143136", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:23.681 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:23.683 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlJHBuSBPhdWcoBPTxDEP5XwPoUUbVrkX1'], u'uuids': [u'2e96e829-1580-4837-a70b-eee2347f9933']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.207812", "end": "2019-12-09 11:17:23.651416", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXlJHBuSBPhdWcoBPTxDEP5XwPoUUbVrkX1"], "labels": [], "masters": [], "uuids": ["2e96e829-1580-4837-a70b-eee2347f9933"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:23.443604", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.201236 s, 573 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.201236 s, 573 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:24.161 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXl3s1coCxfeUjfLN3M6qxOTHsmsrbRa2eZ'], u'uuids': [u'f5546f31-7ee9-4dd9-894e-da07538d4e6e']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.180090", "end": "2019-12-09 11:17:24.133744", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXl3s1coCxfeUjfLN3M6qxOTHsmsrbRa2eZ"], "labels": [], "masters": [], "uuids": ["f5546f31-7ee9-4dd9-894e-da07538d4e6e"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:23.953654", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.173624 s, 664 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.173624 s, 664 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:24.621 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXldS8jYYhqfAjYerBRlZXHA2vQYrvjH2sX'], u'uuids': [u'e054f300-a394-433a-92ad-1241c30d9e48']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT6204013R400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.170238", "end": "2019-12-09 11:17:24.595649", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-7qKR1cEZNI3J3SjY8135Ma4gD9iCshXldS8jYYhqfAjYerBRlZXHA2vQYrvjH2sX"], "labels": [], "masters": [], "uuids": ["e054f300-a394-433a-92ad-1241c30d9e48"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT6204013R400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:24.425411", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.164867 s, 700 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.164867 s, 700 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:24.669 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:24 +0000 (0:00:02.942) 0:04:20.284 ******* 2019-12-09T11:17:24.676 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:24.678 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZngJhLaTdD0KcxjpvoHnfmPyjdycXyzTz'], u'uuids': [u'6e50df23-19d5-4aba-9934-2561af938d89']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.084650", "end": "2019-12-09 11:17:24.606200", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZngJhLaTdD0KcxjpvoHnfmPyjdycXyzTz"], "labels": [], "masters": [], "uuids": ["6e50df23-19d5-4aba-9934-2561af938d89"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:23.521550", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:25.081 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T11:17:25.083 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.083337", "end": "2019-12-09 11:17:25.048706", "rc": 0, "start": "2019-12-09 11:17:24.965369", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:25.117 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.447) 0:04:20.732 ******* 2019-12-09T11:17:25.465 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T11:17:25.467 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.046821", "end": "2019-12-09 11:17:25.438508", "rc": 0, "start": "2019-12-09 11:17:25.391687", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:25.505 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.388) 0:04:21.120 ******* 2019-12-09T11:17:25.532 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T11:17:25.570 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.064) 0:04:21.185 ******* 2019-12-09T11:17:25.596 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T11:17:25.598 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:25.636 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.065) 0:04:21.251 ******* 2019-12-09T11:17:25.663 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T11:17:25.664 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:25.699 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.063) 0:04:21.314 ******* 2019-12-09T11:17:25.728 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T11:17:25.729 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:25.767 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.066) 0:04:21.381 ******* 2019-12-09T11:17:25.803 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T11:17:25.804 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:25.845 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:25 +0000 (0:00:00.078) 0:04:21.460 ******* 2019-12-09T11:17:26.027 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:26.029 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZx80otVP8yX4gHZVTHPBxyelY9rR1ATeO'], u'uuids': [u'1e316c61-b8b1-480d-9a2a-c2e31d9d25e7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.077318", "end": "2019-12-09 11:17:25.988553", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZx80otVP8yX4gHZVTHPBxyelY9rR1ATeO"], "labels": [], "masters": [], "uuids": ["1e316c61-b8b1-480d-9a2a-c2e31d9d25e7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:24.911235", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:26.143 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-09T11:17:26.145 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:17:26.181 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.336) 0:04:21.796 ******* 2019-12-09T11:17:26.218 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T11:17:26.219 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={u'block_used': 13040454, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284030, u'size_available': 930955386880, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284030, "block_size": 4096, "block_total": 240324484, "block_used": 13040454, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955386880, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.259 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.077) 0:04:21.874 ******* 2019-12-09T11:17:26.286 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T11:17:26.287 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.324 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.064) 0:04:21.938 ******* 2019-12-09T11:17:26.349 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:26.350 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.388 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.064) 0:04:22.003 ******* 2019-12-09T11:17:26.414 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:26.416 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.455 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.065) 0:04:22.068 ******* 2019-12-09T11:17:26.491 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T11:17:26.493 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.530 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.077) 0:04:22.145 ******* 2019-12-09T11:17:26.557 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T11:17:26.558 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.593 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.062) 0:04:22.208 ******* 2019-12-09T11:17:26.618 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T11:17:26.620 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:26.656 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:26 +0000 (0:00:00.062) 0:04:22.271 ******* 2019-12-09T11:17:27.427 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T11:17:27.428 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T11:17:27.429 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-12-09T11:17:27.430 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZTQKALNLmVlvuduKIQjrGqgHd0TFcEeDa'], u'uuids': [u'a2dfaa9d-bc34-4e4c-9aa2-a652450df923']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.086272", "end": "2019-12-09 11:17:27.393986", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZTQKALNLmVlvuduKIQjrGqgHd0TFcEeDa"], "labels": [], "masters": [], "uuids": ["a2dfaa9d-bc34-4e4c-9aa2-a652450df923"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:26.307714", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-12-09T11:17:27.468 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:27 +0000 (0:00:00.811) 0:04:23.082 ******* 2019-12-09T11:17:27.536 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:27 +0000 (0:00:00.068) 0:04:23.151 ******* 2019-12-09T11:17:27.584 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:27.585 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XN3Z', u'wwn-0x5000c50092940234'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XN3Z', u'wwn': u'0x5000c50092940234', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XN3Z-part1', u'wwn-0x5000c50092940234-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN3Z", "wwn-0x5000c50092940234"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XN3Z-part1", "wwn-0x5000c50092940234-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XN3Z", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-12-09T11:17:27.587 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c50092940234"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:27.930 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-yUqXsk-Lvso-NkWd-RMSA-nvso-9TqS-ETx7zu', u'nvme-INTEL_SSDPEDMD400G4_PHFT62050074400BGN', u'nvme-nvme.8086-50484654363230353030373434303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'INTEL SSDPEDMD400G4', u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.057184", "end": "2019-12-09 11:17:27.904105", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-yUqXsk-Lvso-NkWd-RMSA-nvso-9TqS-ETx7zu", "nvme-INTEL_SSDPEDMD400G4_PHFT62050074400BGN", "nvme-nvme.8086-50484654363230353030373434303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:27.846921", "stderr": 2019-12-09T11:17:27.932 INFO:teuthology.task.ansible.out: "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.0507266 s, 2.3 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0507266 s, 2.3 GB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:28.239 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:28.241 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:28.420 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:28.421 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZBU8VlgxGI6jhsUtYRY758oViWOUEKbwQ'], u'uuids': [u'58f35930-a2f8-42c9-a909-3d6e78ce0ac1']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.196821", "end": "2019-12-09 11:17:28.379114", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZBU8VlgxGI6jhsUtYRY758oViWOUEKbwQ"], "labels": [], "masters": [], "uuids": ["58f35930-a2f8-42c9-a909-3d6e78ce0ac1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:28.182293", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.190783 s, 605 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.190783 s, 605 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:28.716 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:28.717 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:28.851 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:28.852 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZgr5cws1hcGemrEYZc0BUkl33ji6zD4yq'], u'uuids': [u'44af6425-900b-4501-8c72-82450a536761']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.167059", "end": "2019-12-09 11:17:28.828537", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZgr5cws1hcGemrEYZc0BUkl33ji6zD4yq"], "labels": [], "masters": [], "uuids": ["44af6425-900b-4501-8c72-82450a536761"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:28.661478", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.16202 s, 712 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.16202 s, 712 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:29.132 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:29.134 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:29.351 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:29.352 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZngJhLaTdD0KcxjpvoHnfmPyjdycXyzTz'], u'uuids': [u'6e50df23-19d5-4aba-9934-2561af938d89']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.173292", "end": "2019-12-09 11:17:29.315235", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZngJhLaTdD0KcxjpvoHnfmPyjdycXyzTz"], "labels": [], "masters": [], "uuids": ["6e50df23-19d5-4aba-9934-2561af938d89"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:29.141943", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.167049 s, 690 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.167049 s, 690 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:29.561 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:29.563 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:29.802 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:29.803 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZx80otVP8yX4gHZVTHPBxyelY9rR1ATeO'], u'uuids': [u'1e316c61-b8b1-480d-9a2a-c2e31d9d25e7']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.184319", "end": "2019-12-09 11:17:29.761428", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZx80otVP8yX4gHZVTHPBxyelY9rR1ATeO"], "labels": [], "masters": [], "uuids": ["1e316c61-b8b1-480d-9a2a-c2e31d9d25e7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:29.577109", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.177871 s, 648 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.177871 s, 648 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:30.042 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:30.043 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:30.082 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:30 +0000 (0:00:02.545) 0:04:25.696 ******* 2019-12-09T11:17:30.266 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-12-09T11:17:30.267 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZTQKALNLmVlvuduKIQjrGqgHd0TFcEeDa'], u'uuids': [u'a2dfaa9d-bc34-4e4c-9aa2-a652450df923']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'serial': u'PHFT62050074400BGN', u'holders': [], u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': None, u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.171345", "end": "2019-12-09 11:17:30.228865", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-BUtWsaeQZHQo6yRWJ39B0fOUuQMOAkXZTQKALNLmVlvuduKIQjrGqgHd0TFcEeDa"], "labels": [], "masters": [], "uuids": ["a2dfaa9d-bc34-4e4c-9aa2-a652450df923"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "serial": "PHFT62050074400BGN", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-12-09 11:17:30.057520", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.165162 s, 698 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.165162 s, 698 MB/s"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:30.311 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:30 +0000 (0:00:00.229) 0:04:25.926 ******* 2019-12-09T11:17:30.411 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T11:17:30.412 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T11:17:30.448 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:30 +0000 (0:00:00.137) 0:04:26.063 ******* 2019-12-09T11:17:30.496 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:30.498 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:30.727 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-12-09T11:17:30.728 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.087428", "end": "2019-12-09 11:17:30.691474", "rc": 0, "start": "2019-12-09 11:17:30.604046", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:30.765 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:30 +0000 (0:00:00.317) 0:04:26.380 ******* 2019-12-09T11:17:30.825 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:30.826 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:31.134 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:31.145 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-12-09T11:17:31.146 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.047798", "end": "2019-12-09 11:17:31.109652", "rc": 0, "start": "2019-12-09 11:17:31.061854", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:31.187 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.422) 0:04:26.802 ******* 2019-12-09T11:17:31.216 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-12-09T11:17:31.281 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.093) 0:04:26.896 ******* 2019-12-09T11:17:31.310 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-12-09T11:17:31.312 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:31.352 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.070) 0:04:26.966 ******* 2019-12-09T11:17:31.379 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-12-09T11:17:31.381 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:31.416 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.064) 0:04:27.031 ******* 2019-12-09T11:17:31.443 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T11:17:31.445 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:31.484 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.067) 0:04:27.099 ******* 2019-12-09T11:17:31.490 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:31.491 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:31.514 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-12-09T11:17:31.516 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:31.556 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.072) 0:04:27.171 ******* 2019-12-09T11:17:31.776 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:31.777 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:31.815 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.258) 0:04:27.430 ******* 2019-12-09T11:17:31.884 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-12-09T11:17:31.885 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-12-09T11:17:31.923 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:31 +0000 (0:00:00.107) 0:04:27.538 ******* 2019-12-09T11:17:31.960 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-12-09T11:17:31.961 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={u'block_used': 13040434, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227284050, u'size_available': 930955468800, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227284050, "block_size": 4096, "block_total": 240324484, "block_used": 13040434, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930955468800, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.011 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.087) 0:04:27.626 ******* 2019-12-09T11:17:32.040 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-12-09T11:17:32.041 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.079 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.068) 0:04:27.694 ******* 2019-12-09T11:17:32.107 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:32.108 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.160 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.080) 0:04:27.775 ******* 2019-12-09T11:17:32.167 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T11:17:32.168 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:17:32.209 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.048) 0:04:27.823 ******* 2019-12-09T11:17:32.214 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:32.216 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.255 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.046) 0:04:27.870 ******* 2019-12-09T11:17:32.261 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T11:17:32.264 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T11:17:32.313 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.057) 0:04:27.927 ******* 2019-12-09T11:17:32.319 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-12-09T11:17:32.321 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.358 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.045) 0:04:27.973 ******* 2019-12-09T11:17:32.386 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-12-09T11:17:32.388 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.425 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.066) 0:04:28.040 ******* 2019-12-09T11:17:32.462 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-12-09T11:17:32.463 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:32.498 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:32 +0000 (0:00:00.072) 0:04:28.112 ******* 2019-12-09T11:17:33.022 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-12-09T11:17:33.024 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-12-09T11:17:33.060 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:33 +0000 (0:00:00.562) 0:04:28.675 ******* 2019-12-09T11:17:33.066 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T11:17:33.067 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true} 2019-12-09T11:17:33.109 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:33 +0000 (0:00:00.048) 0:04:28.724 ******* 2019-12-09T11:17:33.536 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:33.537 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:33.891 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T11:17:33.893 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T11:17:33.929 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:33 +0000 (0:00:00.820) 0:04:29.544 ******* 2019-12-09T11:17:33.956 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T11:17:33.958 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:33.994 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:33 +0000 (0:00:00.065) 0:04:29.609 ******* 2019-12-09T11:17:34.001 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:34.002 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:34.024 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T11:17:34.026 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.062 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.068) 0:04:29.677 ******* 2019-12-09T11:17:34.090 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T11:17:34.092 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.127 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.064) 0:04:29.742 ******* 2019-12-09T11:17:34.155 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T11:17:34.156 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.193 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.065) 0:04:29.808 ******* 2019-12-09T11:17:34.221 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T11:17:34.223 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.259 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.065) 0:04:29.873 ******* 2019-12-09T11:17:34.286 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-09T11:17:34.288 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.324 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.065) 0:04:29.939 ******* 2019-12-09T11:17:34.351 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T11:17:34.353 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.389 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.064) 0:04:30.004 ******* 2019-12-09T11:17:34.469 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:34.471 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:34.889 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T11:17:34.890 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T11:17:34.927 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.538) 0:04:30.542 ******* 2019-12-09T11:17:34.954 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T11:17:34.956 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:34.991 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:34 +0000 (0:00:00.064) 0:04:30.606 ******* 2019-12-09T11:17:34.996 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:34.997 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:35.019 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T11:17:35.021 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:35.058 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:35 +0000 (0:00:00.066) 0:04:30.673 ******* 2019-12-09T11:17:35.423 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-12-09T11:17:35.424 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-12-09T11:17:35.464 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:35 +0000 (0:00:00.405) 0:04:31.078 ******* 2019-12-09T11:17:35.808 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-12-09T11:17:35.810 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-12-09T11:17:35.846 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:35 +0000 (0:00:00.382) 0:04:31.461 ******* 2019-12-09T11:17:35.895 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:35.897 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:36.041 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T11:17:36.043 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T11:17:36.084 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:36 +0000 (0:00:00.238) 0:04:31.699 ******* 2019-12-09T11:17:36.111 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T11:17:36.113 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:36.148 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:36 +0000 (0:00:00.064) 0:04:31.763 ******* 2019-12-09T11:17:36.218 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:36.219 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:36.503 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:36.641 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T11:17:36.642 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "basic.target system.slice syslog.target ntpdate.service tmp.mount -.mount systemd-journald.socket sntp.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service multi-user.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T11:17:36.644 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "-.mount basic.target", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T11:17:36.646 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T11:17:36.680 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:36 +0000 (0:00:00.531) 0:04:32.295 ******* 2019-12-09T11:17:36.813 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-12-09T11:17:36.813 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:37.108 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-12-09T11:17:37.150 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.469) 0:04:32.764 ******* 2019-12-09T11:17:37.468 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-12-09T11:17:37.469 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-12-09T11:17:37.507 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.357) 0:04:33.122 ******* 2019-12-09T11:17:37.544 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-12-09T11:17:37.546 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-12-09T11:17:37.582 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.075) 0:04:33.197 ******* 2019-12-09T11:17:37.599 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-09T11:17:37.600 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T11:17:37.639 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.056) 0:04:33.253 ******* 2019-12-09T11:17:37.666 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T11:17:37.668 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:37.704 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.064) 0:04:33.319 ******* 2019-12-09T11:17:37.731 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T11:17:37.733 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:37.768 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:37 +0000 (0:00:00.064) 0:04:33.383 ******* 2019-12-09T11:17:38.047 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-12-09T11:17:38.049 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true} 2019-12-09T11:17:38.085 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.317) 0:04:33.700 ******* 2019-12-09T11:17:38.091 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T11:17:38.091 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T11:17:38.131 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.045) 0:04:33.746 ******* 2019-12-09T11:17:38.512 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-12-09T11:17:38.514 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-12-09T11:17:38.550 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.419) 0:04:34.165 ******* 2019-12-09T11:17:38.576 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2019-12-09T11:17:38.577 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.614 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.063) 0:04:34.229 ******* 2019-12-09T11:17:38.640 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-12-09T11:17:38.642 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.677 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.062) 0:04:34.292 ******* 2019-12-09T11:17:38.703 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-12-09T11:17:38.705 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.741 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.064) 0:04:34.356 ******* 2019-12-09T11:17:38.768 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-12-09T11:17:38.770 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.805 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.063) 0:04:34.420 ******* 2019-12-09T11:17:38.831 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-12-09T11:17:38.833 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.869 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.063) 0:04:34.484 ******* 2019-12-09T11:17:38.895 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-12-09T11:17:38.897 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.934 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.064) 0:04:34.548 ******* 2019-12-09T11:17:38.960 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-12-09T11:17:38.962 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:38.998 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:38 +0000 (0:00:00.064) 0:04:34.613 ******* 2019-12-09T11:17:39.056 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T11:17:39.058 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T11:17:39.093 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:39 +0000 (0:00:00.095) 0:04:34.708 ******* 2019-12-09T11:17:39.489 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-12-09T11:17:39.491 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-12-09T11:17:39.527 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:39 +0000 (0:00:00.433) 0:04:35.142 ******* 2019-12-09T11:17:39.554 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-12-09T11:17:39.556 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:39.592 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:39 +0000 (0:00:00.065) 0:04:35.207 ******* 2019-12-09T11:17:39.619 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2019-12-09T11:17:39.621 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:39.660 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:39 +0000 (0:00:00.064) 0:04:35.272 ******* 2019-12-09T11:17:40.574 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-12-09T11:17:40.576 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-12-09T11:17:40.613 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:40 +0000 (0:00:00.955) 0:04:36.228 ******* 2019-12-09T11:17:40.640 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-12-09T11:17:40.642 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:40.678 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:40 +0000 (0:00:00.064) 0:04:36.293 ******* 2019-12-09T11:17:40.905 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T11:17:40.905 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T11:17:40.949 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:40 +0000 (0:00:00.271) 0:04:36.564 ******* 2019-12-09T11:17:40.979 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T11:17:40.981 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:41.020 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.071) 0:04:36.635 ******* 2019-12-09T11:17:41.058 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-09T11:17:41.096 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.075) 0:04:36.710 ******* 2019-12-09T11:17:41.210 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-12-09T11:17:41.211 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "-.mount tmp.mount sntp.service ntpdate.service syslog.target systemd-journald.socket system.slice basic.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "chronyd.service shutdown.target multi-user.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-12-09T11:17:41.213 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-12-09T11:17:41.215 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-12-09T11:17:41.253 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.156) 0:04:36.867 ******* 2019-12-09T11:17:41.430 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T11:17:41.431 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T11:17:41.468 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.215) 0:04:37.083 ******* 2019-12-09T11:17:41.495 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T11:17:41.497 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:41.533 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.064) 0:04:37.147 ******* 2019-12-09T11:17:41.560 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T11:17:41.562 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T11:17:41.598 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.065) 0:04:37.213 ******* 2019-12-09T11:17:41.625 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T11:17:41.627 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:41.663 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:41 +0000 (0:00:00.064) 0:04:37.278 ******* 2019-12-09T11:17:41.985 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-09T11:17:41.986 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T11:17:42.023 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.360) 0:04:37.638 ******* 2019-12-09T11:17:42.177 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-12-09T11:17:42.179 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-12-09T11:17:42.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.198) 0:04:37.837 ******* 2019-12-09T11:17:42.251 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-12-09T11:17:42.253 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.297 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.074) 0:04:37.912 ******* 2019-12-09T11:17:42.325 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2019-12-09T11:17:42.327 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.367 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.069) 0:04:37.982 ******* 2019-12-09T11:17:42.373 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T11:17:42.374 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T11:17:42.413 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.045) 0:04:38.027 ******* 2019-12-09T11:17:42.441 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T11:17:42.443 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.479 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.066) 0:04:38.094 ******* 2019-12-09T11:17:42.517 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T11:17:42.519 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.531 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.533 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi106.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.543 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.580 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.101) 0:04:38.195 ******* 2019-12-09T11:17:42.618 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-09T11:17:42.620 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.630 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.641 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.653 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.665 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.707 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.126) 0:04:38.321 ******* 2019-12-09T11:17:42.713 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-12-09T11:17:42.714 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-12-09T11:17:42.753 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.046) 0:04:38.368 ******* 2019-12-09T11:17:42.759 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:42.761 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.048) 0:04:38.417 ******* 2019-12-09T11:17:42.831 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T11:17:42.833 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.869 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.066) 0:04:38.483 ******* 2019-12-09T11:17:42.895 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T11:17:42.897 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:42.936 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.067) 0:04:38.551 ******* 2019-12-09T11:17:42.964 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T11:17:42.966 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:43.002 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:42 +0000 (0:00:00.065) 0:04:38.617 ******* 2019-12-09T11:17:43.029 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T11:17:43.031 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:43.069 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:43 +0000 (0:00:00.066) 0:04:38.684 ******* 2019-12-09T11:17:43.096 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T11:17:43.098 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:43.136 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:43 +0000 (0:00:00.067) 0:04:38.751 ******* 2019-12-09T11:17:43.174 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-09T11:17:43.176 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:43.222 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:43 +0000 (0:00:00.085) 0:04:38.837 ******* 2019-12-09T11:17:43.249 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T11:17:43.251 INFO:teuthology.task.ansible.out:skipping: [smithi106.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:43.287 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:43 +0000 (0:00:00.064) 0:04:38.901 ******* 2019-12-09T11:17:43.645 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T11:17:43.646 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016540", "end": "2019-12-09 11:17:43.604245", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 11:17:43.587705", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:43.698 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-12-09T11:17:43.700 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-12-09T11:17:43.737 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:43 +0000 (0:00:00.450) 0:04:39.352 ******* 2019-12-09T11:17:43.971 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T11:17:43.973 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.008350", "end": "2019-12-09 11:17:43.933958", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 11:17:43.925608", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:44.296 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006826", "end": "2019-12-09 11:17:44.260466", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 11:17:44.253640", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:44.590 INFO:teuthology.task.ansible.out:ok: [smithi106.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.007604", "end": "2019-12-09 11:17:44.552518", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 11:17:44.544914", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:44.630 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:44 +0000 (0:00:00.893) 0:04:40.245 ******* 2019-12-09T11:17:44.657 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T11:17:44.695 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:44 +0000 (0:00:00.064) 0:04:40.310 ******* 2019-12-09T11:17:45.036 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-09T11:17:45.037 INFO:teuthology.task.ansible.out:changed: [smithi106.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T11:17:45.354 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-12-09T11:17:45.356 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-12-09T11:17:45.392 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:45 +0000 (0:00:00.696) 0:04:41.007 ******* 2019-12-09T11:17:45.419 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-12-09T11:17:45.421 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:45.458 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:45 +0000 (0:00:00.065) 0:04:41.072 ******* 2019-12-09T11:17:45.485 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-12-09T11:17:45.487 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-12-09T11:17:45.521 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:45 +0000 (0:00:00.063) 0:04:41.136 ******* 2019-12-09T11:17:45.850 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-12-09T11:17:45.852 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-12-09T11:17:45.895 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:45 +0000 (0:00:00.373) 0:04:41.510 ******* 2019-12-09T11:17:45.923 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-12-09T11:17:45.924 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:45.961 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:45 +0000 (0:00:00.065) 0:04:41.576 ******* 2019-12-09T11:17:45.988 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-12-09T11:17:45.989 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-12-09T11:17:46.026 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.065) 0:04:41.641 ******* 2019-12-09T11:17:46.054 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-12-09T11:17:46.056 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.093 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.066) 0:04:41.708 ******* 2019-12-09T11:17:46.412 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-12-09T11:17:46.414 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-12-09T11:17:46.451 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.357) 0:04:42.066 ******* 2019-12-09T11:17:46.761 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-12-09T11:17:46.762 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-12-09T11:17:46.802 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.351) 0:04:42.417 ******* 2019-12-09T11:17:46.829 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-12-09T11:17:46.831 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.870 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.067) 0:04:42.485 ******* 2019-12-09T11:17:46.908 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-12-09T11:17:46.910 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.920 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.921 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.932 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.933 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-12-09T11:17:46.986 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:46 +0000 (0:00:00.115) 0:04:42.601 ******* 2019-12-09T11:17:47.027 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-12-09T11:17:47.028 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.039 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.050 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.051 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.053 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.092 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.105) 0:04:42.707 ******* 2019-12-09T11:17:47.119 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-12-09T11:17:47.121 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.159 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.067) 0:04:42.774 ******* 2019-12-09T11:17:47.186 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-12-09T11:17:47.188 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.224 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.064) 0:04:42.838 ******* 2019-12-09T11:17:47.261 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-12-09T11:17:47.263 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.306 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.082) 0:04:42.921 ******* 2019-12-09T11:17:47.333 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-12-09T11:17:47.335 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.371 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.064) 0:04:42.986 ******* 2019-12-09T11:17:47.397 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-12-09T11:17:47.398 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.437 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.066) 0:04:43.052 ******* 2019-12-09T11:17:47.464 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.508 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.067) 0:04:43.119 ******* 2019-12-09T11:17:47.531 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-12-09T11:17:47.533 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.573 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.068) 0:04:43.188 ******* 2019-12-09T11:17:47.621 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-12-09T11:17:47.623 INFO:teuthology.task.ansible.out:skipping: [smithi174.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-12-09T11:17:47.677 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:47 +0000 (0:00:00.103) 0:04:43.292 ******* 2019-12-09T11:17:48.023 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-12-09T11:17:48.025 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.012052", "end": "2019-12-09 11:17:47.979222", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-12-09 11:17:47.967170", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:48.324 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007035", "end": "2019-12-09 11:17:48.288303", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-12-09 11:17:48.281268", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:48.625 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006878", "end": "2019-12-09 11:17:48.592152", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-12-09 11:17:48.585274", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:48.933 INFO:teuthology.task.ansible.out:ok: [smithi174.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.008733", "end": "2019-12-09 11:17:48.893951", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-12-09 11:17:48.885218", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-12-09T11:17:48.973 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:48 +0000 (0:00:01.296) 0:04:44.588 ******* 2019-12-09T11:17:49.010 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-12-09T11:17:49.046 INFO:teuthology.task.ansible.out:Monday 09 December 2019 11:17:49 +0000 (0:00:00.072) 0:04:44.661 ******* 2019-12-09T11:17:49.350 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-12-09T11:17:49.351 INFO:teuthology.task.ansible.out:changed: [smithi174.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-12-09T11:17:49.451 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** 2019-12-09T11:17:49.453 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2019-12-09T11:17:49.454 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2019-12-09T11:17:49.456 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** 2019-12-09T11:17:49.457 INFO:teuthology.task.ansible.out:skipping: no hosts matched PLAY RECAP ********************************************************************* smithi106.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 smithi174.front.sepia.ceph.com : ok=121 changed=41 unreachable=0 failed=0 skipped=160 rescued=0 ignored=3 Monday 09 December 2019 11:17:49 +0000 (0:00:00.410) 0:04:45.071 ******* =============================================================================== 2019-12-09T11:17:49.459 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 34.76s users : Update authorized_keys using the keys repo --------------------- 33.57s common : nrpe - Ensure SELinux policy is up to date -------------------- 22.31s users : Remove revoked users ------------------------------------------- 14.67s testnode : Install packages -------------------------------------------- 12.53s common : Update yum cache ----------------------------------------------- 9.69s testnode : Ensure ceph dependency packages are not present. ------------- 3.74s Gathering Facts --------------------------------------------------------- 3.27s testnode : Zap all non-root disks --------------------------------------- 3.20s testnode : Ensure ceph packages are not present. ------------------------ 3.18s common : nrpe - Remove SELinux policy package --------------------------- 2.89s ansible-managed : Add authorized keys for the ansible user. ------------- 2.73s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 2.62s common : nrpe - Load SELinux policy package ----------------------------- 2.60s testnode : Blow away lingering OSD data and FSIDs ----------------------- 2.55s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.87s common : Configure epel repos in /etc/yum.repos.d/ ---------------------- 1.87s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 1.86s ansible-managed : Create the cephlab_sudo sudoers.d file. --------------- 1.45s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.30s 2019-12-09T11:17:49.723 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2019-12-09T11:17:49.726 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:17:49.726 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi106.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:17:49.824 INFO:teuthology.orchestra.run.smithi106:> true 2019-12-09T11:17:50.214 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-12-09T11:17:50.215 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi174.front.sepia.ceph.com', 'timeout': 60} 2019-12-09T11:17:50.314 INFO:teuthology.orchestra.run.smithi174:> true 2019-12-09T11:17:50.689 INFO:teuthology.run_tasks:Running task clock... 2019-12-09T11:17:50.707 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-12-09T11:17:50.708 INFO:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T11:17:50.843 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T11:18:00.924 INFO:teuthology.orchestra.run.smithi106.stdout:ntpd: time slew -0.000802s 2019-12-09T11:18:00.962 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================== 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:01.048 INFO:teuthology.orchestra.run.smithi106.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:01.049 INFO:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-12-09T11:18:01.187 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-12-09T11:18:11.283 INFO:teuthology.orchestra.run.smithi174.stdout:ntpd: time slew +0.001763s 2019-12-09T11:18:11.321 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit not found. 2019-12-09T11:18:11.396 INFO:teuthology.orchestra.run.smithi174.stdout: remote refid st t when poll reach delay offset jitter 2019-12-09T11:18:11.396 INFO:teuthology.orchestra.run.smithi174.stdout:============================================================================== 2019-12-09T11:18:11.396 INFO:teuthology.orchestra.run.smithi174.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:11.396 INFO:teuthology.orchestra.run.smithi174.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:11.397 INFO:teuthology.orchestra.run.smithi174.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:11.397 INFO:teuthology.orchestra.run.smithi174.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-12-09T11:18:11.398 INFO:teuthology.run_tasks:Running task install... 2019-12-09T11:18:11.408 DEBUG:teuthology.task.install:project ceph 2019-12-09T11:18:11.409 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}} 2019-12-09T11:18:11.409 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'} 2019-12-09T11:18:11.409 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-12-09T11:18:11.413 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-12-09T11:18:11.413 INFO:teuthology.task.install:extra packages: [] 2019-12-09T11:18:11.414 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2019-12-09T11:18:11.580 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS 2019-12-09T11:18:11.584 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:18:11.584 INFO:teuthology.orchestra.run.smithi174:> sudo lsb_release -is 2019-12-09T11:18:11.585 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T11:18:11.585 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2019-12-09T11:18:11.709 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS 2019-12-09T11:18:11.711 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:18:11.712 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:18:11.745 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS 2019-12-09T11:18:11.745 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:18:11.746 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-12-09T11:18:11.746 INFO:teuthology.orchestra.run.smithi174:> sudo lsb_release -is 2019-12-09T11:18:11.867 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T11:18:11.867 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T11:18:11.882 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS 2019-12-09T11:18:11.884 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:18:11.884 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:18:11.986 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T11:18:12.035 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-ofriedma-dynamic-disable/1351c819e1a9518cde4c8b12d645782009028360/centos/7/flavors/notcmalloc/ 2019-12-09T11:18:12.035 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7900.g1351c81 2019-12-09T11:18:12.106 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T11:18:12.106 INFO:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T11:18:12.168 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-12-09T11:18:12.212 INFO:teuthology.orchestra.run.smithi106:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T11:18:12.287 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-12-09T11:18:12.288 INFO:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1351c819e1a9518cde4c8b12d645782009028360/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-12-09T11:18:12.295 INFO:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T11:18:12.373 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2019-12-09T11:18:12.376 INFO:teuthology.orchestra.run.smithi106:> sudo yum clean all 2019-12-09T11:18:12.401 INFO:teuthology.orchestra.run.smithi174:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-12-09T11:18:12.512 INFO:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-12-09T11:18:12.542 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:12.580 INFO:teuthology.orchestra.run.smithi106.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T11:18:12.580 INFO:teuthology.orchestra.run.smithi106.stdout: : lab-extras updates 2019-12-09T11:18:12.620 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2019-12-09T11:18:12.622 INFO:teuthology.orchestra.run.smithi174:> sudo yum clean all 2019-12-09T11:18:12.627 INFO:teuthology.orchestra.run.smithi106.stdout:Cleaning up list of fastest mirrors 2019-12-09T11:18:12.628 INFO:teuthology.orchestra.run.smithi106.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T11:18:12.639 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw 2019-12-09T11:18:12.796 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:12.833 INFO:teuthology.orchestra.run.smithi106.stdout:Determining fastest mirrors 2019-12-09T11:18:12.854 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:12.896 INFO:teuthology.orchestra.run.smithi174.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-12-09T11:18:12.896 INFO:teuthology.orchestra.run.smithi174.stdout: : lab-extras updates 2019-12-09T11:18:12.942 INFO:teuthology.orchestra.run.smithi174.stdout:Cleaning up list of fastest mirrors 2019-12-09T11:18:12.943 INFO:teuthology.orchestra.run.smithi174.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-12-09T11:18:12.956 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw 2019-12-09T11:18:13.131 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:18:13.132 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:18:13.132 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:18:13.133 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:18:13.190 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:13.232 INFO:teuthology.orchestra.run.smithi174.stdout:Determining fastest mirrors 2019-12-09T11:18:13.776 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:18:13.777 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:18:13.777 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:18:13.778 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:18:20.701 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:18:21.053 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:18:21.053 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:21.053 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.066 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.171 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.173 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.174 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.176 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.176 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.178 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.178 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.180 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:21.181 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.189 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.190 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.191 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.193 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.194 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.197 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T11:18:21.198 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T11:18:21.198 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T11:18:21.198 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T11:18:21.200 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T11:18:21.201 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.202 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:21.202 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.206 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.207 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.208 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.209 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.210 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.211 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.212 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.213 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.214 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.214 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.215 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:21.215 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T11:18:21.216 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.216 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T11:18:21.216 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T11:18:21.217 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T11:18:21.217 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T11:18:21.219 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T11:18:21.219 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:21.219 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T11:18:21.220 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.221 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.221 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.221 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.222 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T11:18:21.223 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.223 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.223 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:21.223 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T11:18:21.827 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:18:22.079 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:18:22.165 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:22.165 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:22.168 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T11:18:22.169 INFO:teuthology.orchestra.run.smithi106.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T11:18:22.170 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T11:18:22.171 INFO:teuthology.orchestra.run.smithi106.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout:Updating for dependencies: 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T11:18:22.172 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:22.173 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 45 M 2019-12-09T11:18:22.174 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:18:22.174 INFO:teuthology.orchestra.run.smithi106.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T11:18:22.184 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:18:22.184 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:22.184 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.197 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.303 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.305 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.306 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.307 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.308 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.310 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.310 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.311 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.312 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:22.313 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.321 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.322 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.323 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.325 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.325 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.326 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.329 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-12-09T11:18:22.330 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-12-09T11:18:22.330 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T11:18:22.330 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librados2.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T11:18:22.332 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-12-09T11:18:22.333 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.334 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:22.334 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.339 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.340 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.341 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.341 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.342 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.343 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.344 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.345 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.346 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.346 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.347 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:22.348 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-12-09T11:18:22.348 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.348 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-12-09T11:18:22.348 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 will be an update 2019-12-09T11:18:22.349 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-12-09T11:18:22.350 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T11:18:22.351 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-12-09T11:18:22.351 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:22.351 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-12-09T11:18:22.352 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.353 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.353 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.353 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.354 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-12-09T11:18:22.355 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.355 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.355 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:22.356 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-12-09T11:18:23.214 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:18:23.300 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:23.300 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:18:23.303 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:23.303 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:23.303 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:18:23.303 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:18:23.304 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:23.304 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:18:23.304 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 6.2 M 2019-12-09T11:18:23.304 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:18:23.304 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.6 M 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 M 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:15.0.0-7900.g1351c81.el7 ceph 20 k 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 489 k 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-12-09T11:18:23.305 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 374 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 5.4 M 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-argparse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 35 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-common x86_64 2:15.0.0-7900.g1351c81.el7 ceph 42 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: python-cephfs x86_64 2:15.0.0-7900.g1351c81.el7 ceph 102 k 2019-12-09T11:18:23.306 INFO:teuthology.orchestra.run.smithi174.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: python-rados x86_64 2:15.0.0-7900.g1351c81.el7 ceph 209 k 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: python-rbd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 199 k 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: python-rgw x86_64 2:15.0.0-7900.g1351c81.el7 ceph 77 k 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout:Updating for dependencies: 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 3.5 M 2019-12-09T11:18:23.307 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.8 M 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+20 Dependent packages) 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade ( 2 Dependent packages) 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:23.308 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 45 M 2019-12-09T11:18:23.309 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:18:23.309 INFO:teuthology.orchestra.run.smithi174.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-12-09T11:18:24.368 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:18:24.368 INFO:teuthology.orchestra.run.smithi106.stdout:Total 20 MB/s | 45 MB 00:02 2019-12-09T11:18:24.381 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:18:24.442 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:18:24.669 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:18:24.669 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:18:25.305 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T11:18:25.667 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T11:18:26.016 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T11:18:26.320 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:18:26.321 INFO:teuthology.orchestra.run.smithi174.stdout:Total 15 MB/s | 45 MB 00:03 2019-12-09T11:18:26.334 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:18:26.370 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T11:18:26.398 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:18:26.630 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:18:26.631 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:18:26.652 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T11:18:27.004 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T11:18:27.400 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-12-09T11:18:27.751 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-12-09T11:18:27.769 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T11:18:28.102 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-12-09T11:18:28.156 INFO:teuthology.orchestra.run.smithi106.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T11:18:28.464 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T11:18:28.749 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T11:18:29.048 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T11:18:29.150 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-12-09T11:18:29.407 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T11:18:29.961 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T11:18:29.983 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-12-09T11:18:30.255 INFO:teuthology.orchestra.run.smithi106.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T11:18:30.396 INFO:teuthology.orchestra.run.smithi174.stdout: Updating : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 8/25 2019-12-09T11:18:30.507 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T11:18:30.758 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T11:18:31.072 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T11:18:31.281 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T11:18:31.355 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T11:18:31.666 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T11:18:31.689 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T11:18:32.034 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T11:18:32.266 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 11/25 2019-12-09T11:18:32.611 INFO:teuthology.orchestra.run.smithi174.stdout: Updating : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T11:18:32.926 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 13/25 2019-12-09T11:18:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 14/25 2019-12-09T11:18:33.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T11:18:33.869 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T11:18:34.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-12-09T11:18:34.471 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T11:18:34.527 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 18/25 2019-12-09T11:18:35.620 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T11:18:36.157 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T11:18:36.981 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 19/25 2019-12-09T11:18:38.152 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T11:18:38.751 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 21/25 2019-12-09T11:18:45.491 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T11:18:45.887 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T11:18:46.351 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T11:18:46.661 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T11:18:46.800 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T11:18:46.909 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T11:18:47.043 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T11:18:47.152 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T11:18:47.252 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T11:18:47.361 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T11:18:47.478 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T11:18:47.579 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T11:18:47.671 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T11:18:47.755 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T11:18:47.872 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T11:18:48.006 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T11:18:48.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T11:18:48.131 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T11:18:48.240 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T11:18:48.366 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T11:18:48.474 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T11:18:48.584 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T11:18:48.600 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T11:18:48.725 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T11:18:48.834 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T11:18:48.896 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T11:18:48.945 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T11:18:49.046 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T11:18:49.130 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T11:18:49.216 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T11:18:49.272 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T11:18:49.273 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T11:18:49.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7900.g1351c81.el7.x86_64 1/25 2019-12-09T11:18:49.443 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-12-09T11:18:49.560 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:libradosstriper1-15.0.0-7900.g1351c81.el7.x86_64 3/25 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:49.654 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.655 INFO:teuthology.orchestra.run.smithi106.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T11:18:49.656 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.657 INFO:teuthology.orchestra.run.smithi106.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Updated: 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:49.658 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:18:49.693 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-12-09T11:18:49.773 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-test 2019-12-09T11:18:49.810 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 5/25 2019-12-09T11:18:49.969 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-ceph-common-15.0.0-7900.g1351c81.el7.x86_64 6/25 2019-12-09T11:18:50.078 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:librgw2-15.0.0-7900.g1351c81.el7.x86_64 7/25 2019-12-09T11:18:50.085 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:50.130 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:18:50.131 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:18:50.131 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:18:50.131 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:18:50.132 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:18:50.229 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 8/25 2019-12-09T11:18:50.346 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-common-15.0.0-7900.g1351c81.el7.x86_64 9/25 2019-12-09T11:18:50.496 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 10/25 2019-12-09T11:18:50.622 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 11/25 2019-12-09T11:18:50.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-rgw-15.0.0-7900.g1351c81.el7.x86_64 12/25 2019-12-09T11:18:50.776 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:18:50.881 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 13/25 2019-12-09T11:18:51.007 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 14/25 2019-12-09T11:18:51.082 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-selinux-15.0.0-7900.g1351c81.el7.x86_64 15/25 2019-12-09T11:18:51.112 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:18:51.112 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:51.112 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:51.130 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:51.166 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-radosgw-15.0.0-7900.g1351c81.el7.x86_64 16/25 2019-12-09T11:18:51.214 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:51.215 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:51.217 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:51.217 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T11:18:51.218 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T11:18:51.219 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T11:18:51.222 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T11:18:51.225 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:18:51.225 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T11:18:51.300 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 17/25 2019-12-09T11:18:51.384 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-rbd-15.0.0-7900.g1351c81.el7.x86_64 18/25 2019-12-09T11:18:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 19/25 2019-12-09T11:18:51.600 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:18:51.618 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-base-15.0.0-7900.g1351c81.el7.x86_64 20/25 2019-12-09T11:18:51.767 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-12-09T11:18:51.818 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:51.819 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:51.820 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T11:18:51.821 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 38 M 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 134 M 2019-12-09T11:18:51.822 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:18:51.877 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-rados-15.0.0-7900.g1351c81.el7.x86_64 22/25 2019-12-09T11:18:51.995 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-cephfs-15.0.0-7900.g1351c81.el7.x86_64 23/25 2019-12-09T11:18:51.996 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-12-09T11:18:52.419 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.420 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-12-09T11:18:52.421 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-argparse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-common.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.422 INFO:teuthology.orchestra.run.smithi174.stdout: python-cephfs.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: python-rados.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: python-rbd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: python-rgw.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:52.423 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Updated: 2019-12-09T11:18:52.424 INFO:teuthology.orchestra.run.smithi174.stdout: librados2.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.424 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:18:52.424 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:52.424 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:18:52.545 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-test 2019-12-09T11:18:52.789 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:18:52.831 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:18:52.832 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:18:52.833 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:18:52.833 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:18:52.833 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:18:53.494 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:18:53.626 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:18:53.626 INFO:teuthology.orchestra.run.smithi106.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-09T11:18:53.629 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:18:53.650 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:18:53.751 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:18:53.751 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:18:53.827 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:18:53.828 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:53.828 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:18:53.845 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:53.875 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:53.876 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:18:53.877 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:53.877 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-12-09T11:18:53.878 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-12-09T11:18:53.879 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-12-09T11:18:53.881 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-12-09T11:18:53.883 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:18:53.883 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-12-09T11:18:54.246 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:18:54.415 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T11:18:54.469 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:54.470 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:18:54.471 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:54.471 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:54.471 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:18:54.471 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:18:54.471 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:15.0.0-7900.g1351c81.el7 ceph 38 M 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-12-09T11:18:54.472 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:18:54.473 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 38 M 2019-12-09T11:18:54.474 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 134 M 2019-12-09T11:18:54.474 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:18:54.737 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T11:18:55.031 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T11:18:56.268 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:18:56.268 INFO:teuthology.orchestra.run.smithi174.stdout:Total 21 MB/s | 38 MB 00:01 2019-12-09T11:18:56.270 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:18:56.288 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:18:56.365 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:18:56.365 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:18:57.113 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-12-09T11:18:57.417 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-12-09T11:18:57.761 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-12-09T11:18:59.018 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T11:18:59.330 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T11:18:59.446 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T11:18:59.555 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T11:18:59.689 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T11:18:59.797 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T11:19:00.305 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T11:19:00.305 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:00.305 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:00.306 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:00.419 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph 2019-12-09T11:19:00.648 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:00.691 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:00.692 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:00.692 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:00.692 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:00.693 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:01.337 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:01.669 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:01.669 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.669 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:01.671 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.701 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.702 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.703 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.704 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.705 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:01.714 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:01.716 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.717 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.719 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.720 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.721 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.722 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:01.723 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:01.725 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:01.730 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.730 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T11:19:01.730 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T11:19:01.731 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T11:19:01.732 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:01.733 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:01.734 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:01.734 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:01.735 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:01.736 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T11:19:01.736 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T11:19:01.737 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T11:19:01.738 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T11:19:01.738 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T11:19:01.739 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.739 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T11:19:01.739 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T11:19:01.740 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T11:19:01.741 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T11:19:01.741 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T11:19:01.741 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T11:19:01.742 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T11:19:01.742 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T11:19:01.742 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T11:19:01.743 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.743 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T11:19:01.743 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T11:19:01.744 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T11:19:01.744 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.744 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T11:19:01.744 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T11:19:01.745 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:01.745 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T11:19:01.833 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-12-09T11:19:02.106 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:02.187 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T11:19:02.326 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:02.326 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:02.329 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:02.329 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:02.329 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:02.330 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T11:19:02.331 INFO:teuthology.orchestra.run.smithi106.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T11:19:02.332 INFO:teuthology.orchestra.run.smithi106.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T11:19:02.332 INFO:teuthology.orchestra.run.smithi106.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T11:19:02.332 INFO:teuthology.orchestra.run.smithi106.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T11:19:02.333 INFO:teuthology.orchestra.run.smithi106.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T11:19:02.334 INFO:teuthology.orchestra.run.smithi106.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T11:19:02.334 INFO:teuthology.orchestra.run.smithi106.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T11:19:02.334 INFO:teuthology.orchestra.run.smithi106.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T11:19:02.334 INFO:teuthology.orchestra.run.smithi106.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T11:19:02.334 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:02.335 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:02.335 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:02.335 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T11:19:02.335 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:02.336 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-test-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T11:19:02.337 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 28 M 2019-12-09T11:19:02.337 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 87 M 2019-12-09T11:19:02.337 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:02.444 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 2/5 2019-12-09T11:19:02.570 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 3/5 2019-12-09T11:19:03.162 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : jq-1.5-1.el7.x86_64 4/5 2019-12-09T11:19:03.605 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 5/5 2019-12-09T11:19:03.605 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:03.605 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:03.605 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:03.605 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:03.606 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:19:03.606 INFO:teuthology.orchestra.run.smithi174.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-12-09T11:19:03.606 INFO:teuthology.orchestra.run.smithi174.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-12-09T11:19:03.606 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:03.606 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:03.670 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:03.670 INFO:teuthology.orchestra.run.smithi106.stdout:Total 21 MB/s | 28 MB 00:01 2019-12-09T11:19:03.684 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:03.713 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:03.733 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph 2019-12-09T11:19:03.969 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:03.984 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:03.985 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:04.008 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:04.012 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:04.012 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:04.012 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:04.012 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:04.681 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:04.683 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T11:19:05.001 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T11:19:05.007 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:05.007 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.008 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:05.010 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.040 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.041 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.042 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.043 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.044 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:05.053 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:05.055 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.056 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.058 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.059 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.060 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.061 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:05.062 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:05.064 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:05.069 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.069 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-12-09T11:19:05.070 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-12-09T11:19:05.071 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-12-09T11:19:05.071 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:05.072 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:05.073 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:05.074 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:05.075 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-12-09T11:19:05.075 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-12-09T11:19:05.076 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-12-09T11:19:05.077 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-12-09T11:19:05.077 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-12-09T11:19:05.078 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-12-09T11:19:05.079 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.079 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-12-09T11:19:05.079 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T11:19:05.079 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-12-09T11:19:05.080 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-12-09T11:19:05.081 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-12-09T11:19:05.081 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-12-09T11:19:05.082 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-12-09T11:19:05.083 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-12-09T11:19:05.083 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.083 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-12-09T11:19:05.084 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-12-09T11:19:05.084 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:05.084 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-12-09T11:19:05.301 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T11:19:05.441 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:05.664 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:05.664 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:05.667 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:05.667 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:05.667 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:19:05.667 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.0 k 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.9 M 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:15.0.0-7900.g1351c81.el7 ceph 1.7 M 2019-12-09T11:19:05.668 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 4.4 M 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:15.0.0-7900.g1351c81.el7 ceph 16 M 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-12-09T11:19:05.669 INFO:teuthology.orchestra.run.smithi174.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-12-09T11:19:05.670 INFO:teuthology.orchestra.run.smithi174.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:05.671 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:05.672 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+21 Dependent packages) 2019-12-09T11:19:05.672 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:05.672 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 28 M 2019-12-09T11:19:05.673 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 87 M 2019-12-09T11:19:05.673 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:05.780 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T11:19:06.169 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T11:19:07.104 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:07.104 INFO:teuthology.orchestra.run.smithi174.stdout:Total 19 MB/s | 28 MB 00:01 2019-12-09T11:19:07.118 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:07.145 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:07.345 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T11:19:07.418 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:07.419 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:07.845 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T11:19:08.546 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-12-09T11:19:08.931 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-12-09T11:19:09.247 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-12-09T11:19:09.740 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T11:19:09.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-12-09T11:19:10.124 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T11:19:10.183 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-12-09T11:19:10.508 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T11:19:10.651 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-12-09T11:19:10.825 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T11:19:11.169 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-12-09T11:19:11.548 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T11:19:12.005 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T11:19:12.377 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T11:19:12.929 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T11:19:13.092 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-12-09T11:19:13.273 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T11:19:13.473 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T11:19:13.536 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 9/22 2019-12-09T11:19:13.711 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T11:19:13.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 10/22 2019-12-09T11:19:14.104 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T11:19:14.280 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-mako-0.8.1-2.el7.noarch 11/22 2019-12-09T11:19:14.743 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T11:19:15.034 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 12/22 2019-12-09T11:19:15.150 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T11:19:15.429 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T11:19:15.500 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T11:19:15.532 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T11:19:15.616 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T11:19:15.716 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T11:19:15.833 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T11:19:15.882 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-12-09T11:19:15.950 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T11:19:16.051 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T11:19:16.185 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T11:19:16.302 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T11:19:16.428 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T11:19:16.488 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-12-09T11:19:16.536 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T11:19:16.837 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T11:19:16.905 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 16/22 2019-12-09T11:19:16.962 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T11:19:17.071 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T11:19:17.110 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-12-09T11:19:17.197 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T11:19:17.322 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T11:19:17.422 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T11:19:17.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-12-09T11:19:17.540 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T11:19:17.640 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T11:19:17.757 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T11:19:17.833 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T11:19:17.842 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-12-09T11:19:17.917 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T11:19:18.361 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:18.362 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T11:19:18.363 INFO:teuthology.orchestra.run.smithi106.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T11:19:18.364 INFO:teuthology.orchestra.run.smithi106.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T11:19:18.365 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:18.366 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:18.479 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-daemon 2019-12-09T11:19:18.574 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-12-09T11:19:18.772 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:18.818 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:18.819 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:18.820 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:18.820 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:18.821 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:19.037 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 21/22 2019-12-09T11:19:19.327 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 22/22 2019-12-09T11:19:19.436 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-12-09T11:19:19.464 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:19.519 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-12-09T11:19:19.637 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-12-09T11:19:19.720 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 4/22 2019-12-09T11:19:19.802 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:19.803 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:19.803 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:19.805 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mds-15.0.0-7900.g1351c81.el7.x86_64 5/22 2019-12-09T11:19:19.938 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 6/22 2019-12-09T11:19:20.047 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mon-15.0.0-7900.g1351c81.el7.x86_64 7/22 2019-12-09T11:19:20.165 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:20.180 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-15.0.0-7900.g1351c81.el7.x86_64 8/22 2019-12-09T11:19:20.298 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 9/22 2019-12-09T11:19:20.572 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:20.572 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:20.572 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:20.572 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:20.572 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:20.573 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:20.575 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:20.575 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2019-12-09T11:19:20.575 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:20.575 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 17 k 2019-12-09T11:19:20.575 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 68 k 2019-12-09T11:19:20.576 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:20.576 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 10/22 2019-12-09T11:19:20.576 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-12-09T11:19:20.679 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:20.687 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:20.689 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:20.690 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:20.691 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-12-09T11:19:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-osd-15.0.0-7900.g1351c81.el7.x86_64 13/22 2019-12-09T11:19:20.930 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 14/22 2019-12-09T11:19:21.067 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 15/22 2019-12-09T11:19:21.168 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-12-09T11:19:21.318 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-12-09T11:19:21.418 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 18/22 2019-12-09T11:19:21.511 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:19:21.535 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 19/22 2019-12-09T11:19:21.619 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 20/22 2019-12-09T11:19:21.703 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 21/22 2019-12-09T11:19:21.899 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:19:21.900 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:21.900 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:21.900 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:21.900 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:21.900 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:22.023 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr 2019-12-09T11:19:22.156 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-12-09T11:19:22.156 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:22.156 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:22.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:22.157 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:22.157 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:19:22.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:22.157 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:22.158 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:22.158 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:22.158 INFO:teuthology.orchestra.run.smithi174.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-12-09T11:19:22.158 INFO:teuthology.orchestra.run.smithi174.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-12-09T11:19:22.159 INFO:teuthology.orchestra.run.smithi174.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-12-09T11:19:22.159 INFO:teuthology.orchestra.run.smithi174.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-12-09T11:19:22.159 INFO:teuthology.orchestra.run.smithi174.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-12-09T11:19:22.159 INFO:teuthology.orchestra.run.smithi174.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-12-09T11:19:22.159 INFO:teuthology.orchestra.run.smithi174.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-12-09T11:19:22.160 INFO:teuthology.orchestra.run.smithi174.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-12-09T11:19:22.160 INFO:teuthology.orchestra.run.smithi174.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-12-09T11:19:22.160 INFO:teuthology.orchestra.run.smithi174.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-12-09T11:19:22.160 INFO:teuthology.orchestra.run.smithi174.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-12-09T11:19:22.161 INFO:teuthology.orchestra.run.smithi174.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-12-09T11:19:22.161 INFO:teuthology.orchestra.run.smithi174.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-12-09T11:19:22.161 INFO:teuthology.orchestra.run.smithi174.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-12-09T11:19:22.161 INFO:teuthology.orchestra.run.smithi174.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-12-09T11:19:22.161 INFO:teuthology.orchestra.run.smithi174.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-12-09T11:19:22.162 INFO:teuthology.orchestra.run.smithi174.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-12-09T11:19:22.162 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:22.162 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:22.271 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-daemon 2019-12-09T11:19:22.276 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:22.316 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:22.317 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:22.317 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:22.318 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:22.318 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:22.558 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:22.603 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:22.604 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:22.604 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:22.604 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:22.605 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:22.968 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:23.261 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:23.300 INFO:teuthology.orchestra.run.smithi106.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:19:23.301 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do 2019-12-09T11:19:23.361 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T11:19:23.578 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:23.600 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:23.601 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:23.601 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:23.622 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:23.623 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:23.624 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:23.624 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:23.624 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:23.966 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:24.189 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:24.189 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:24.190 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:24.190 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:24.190 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-daemon x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:24.191 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:24.192 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T11:19:24.192 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:24.192 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 17 k 2019-12-09T11:19:24.192 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 68 k 2019-12-09T11:19:24.192 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:24.268 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:24.500 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:24.508 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:24.510 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:24.510 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:24.601 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:24.602 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:24.602 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:24.605 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:24.634 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:24.635 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:24.636 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:24.636 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T11:19:24.636 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T11:19:24.637 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:24.637 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T11:19:25.001 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:25.222 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:25.222 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:25.223 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:25.223 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:25.223 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T11:19:25.224 INFO:teuthology.orchestra.run.smithi106.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 4.6 M 2019-12-09T11:19:25.225 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 20 M 2019-12-09T11:19:25.226 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:25.421 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:19:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:Total 7.2 MB/s | 4.6 MB 00:00 2019-12-09T11:19:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:25.886 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-daemon-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:19:25.886 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:25.887 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:25.887 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-daemon.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:25.887 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:25.887 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:26.006 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr 2019-12-09T11:19:26.033 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:26.033 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:26.255 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:26.297 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:26.297 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:26.298 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:26.298 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:26.299 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:26.959 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:27.083 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T11:19:27.290 INFO:teuthology.orchestra.run.smithi174.stdout:Package 2:ceph-mgr-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:19:27.290 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do 2019-12-09T11:19:27.353 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-dashboard 2019-12-09T11:19:27.565 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:27.609 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:27.610 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:27.610 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:27.611 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:27.611 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:27.816 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T11:19:28.266 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:28.603 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:28.604 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:28.604 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:28.607 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7900.g1351c81.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:28.637 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:28.638 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:28.639 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:28.639 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-12-09T11:19:28.639 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-12-09T11:19:28.640 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:28.640 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-12-09T11:19:28.947 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T11:19:29.005 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:29.230 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:29.230 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:29.231 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:29.231 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:29.231 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 3.9 M 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 18 k 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-12-09T11:19:29.232 INFO:teuthology.orchestra.run.smithi174.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+3 Dependent packages) 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 4.6 M 2019-12-09T11:19:29.233 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 20 M 2019-12-09T11:19:29.234 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:29.361 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T11:19:29.495 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T11:19:29.646 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T11:19:29.746 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T11:19:29.850 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:29.851 INFO:teuthology.orchestra.run.smithi174.stdout:Total 7.4 MB/s | 4.6 MB 00:00 2019-12-09T11:19:29.857 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:29.866 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:30.030 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:30.031 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:30.148 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:30.149 INFO:teuthology.orchestra.run.smithi106.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T11:19:30.150 INFO:teuthology.orchestra.run.smithi106.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T11:19:30.150 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:30.150 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:30.282 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T11:19:30.529 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:30.569 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:30.570 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:30.570 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:30.570 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:30.571 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:30.656 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 1/4 2019-12-09T11:19:31.110 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-12-09T11:19:31.218 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:31.546 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:31.546 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:31.547 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:31.909 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:32.131 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:32.131 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:32.132 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:32.132 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:32.133 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 82 k 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 411 k 2019-12-09T11:19:32.134 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:32.281 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-12-09T11:19:32.439 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:32.443 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:32.498 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:32.498 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:32.796 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 4/4 2019-12-09T11:19:33.196 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T11:19:33.701 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T11:19:33.702 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:33.702 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:33.702 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:33.702 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:33.703 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:33.821 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T11:19:33.948 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-12-09T11:19:34.090 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7900.g1351c81.el7.noarch 2/4 2019-12-09T11:19:34.092 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:34.139 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:34.140 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:34.141 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:34.141 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:34.141 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:34.240 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7900.g1351c81.el7.noarch 3/4 2019-12-09T11:19:34.725 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-routes-1.13-2.el7.noarch 4/4 2019-12-09T11:19:34.725 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:34.725 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-12-09T11:19:34.726 INFO:teuthology.orchestra.run.smithi174.stdout: python-routes.noarch 0:1.13-2.el7 2019-12-09T11:19:34.727 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:34.727 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:34.792 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:34.847 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-12-09T11:19:35.109 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:35.135 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:35.136 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:35.136 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:35.138 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:35.154 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:35.155 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:35.155 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:35.155 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:35.156 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:35.168 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:35.169 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T11:19:35.175 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:35.176 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:35.178 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:35.178 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:35.179 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T11:19:35.179 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T11:19:35.180 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T11:19:35.182 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T11:19:35.182 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:35.182 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T11:19:35.545 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:35.766 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:35.766 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:35.767 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:35.767 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:35.768 INFO:teuthology.orchestra.run.smithi106.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:35.769 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:19:35.770 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:35.770 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 19 M 2019-12-09T11:19:35.770 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 104 M 2019-12-09T11:19:35.770 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:35.806 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:36.146 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:36.147 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:36.147 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:36.491 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:36.491 INFO:teuthology.orchestra.run.smithi106.stdout:Total 26 MB/s | 19 MB 00:00 2019-12-09T11:19:36.503 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:36.511 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:36.532 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:36.708 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:36.708 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:36.735 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:36.735 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:36.736 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:36.736 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:36.736 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-cloud 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 82 k 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:36.737 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:36.738 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T11:19:36.738 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:36.738 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 82 k 2019-12-09T11:19:36.738 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 411 k 2019-12-09T11:19:36.738 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:37.046 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:37.050 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:37.105 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:37.105 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:37.553 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T11:19:37.731 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T11:19:38.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T11:19:38.261 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7900.g1351c81.el7.n 1/1 2019-12-09T11:19:38.261 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:38.261 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:38.261 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:38.262 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:38.262 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:38.377 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-diskprediction-local 2019-12-09T11:19:38.641 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:38.684 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:38.685 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:38.686 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:38.686 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:38.687 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:39.335 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:39.676 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:39.676 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:39.677 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:39.679 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:39.709 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:39.710 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-12-09T11:19:39.716 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:39.717 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:39.719 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-12-09T11:19:39.719 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:39.719 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-12-09T11:19:39.720 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-12-09T11:19:39.721 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T11:19:39.723 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-12-09T11:19:39.723 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:39.723 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-12-09T11:19:39.731 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T11:19:40.084 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:40.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:40.309 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:40.310 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:40.310 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:40.310 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:19:40.310 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:40.310 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 8.6 M 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-12-09T11:19:40.311 INFO:teuthology.orchestra.run.smithi174.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 19 M 2019-12-09T11:19:40.312 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 104 M 2019-12-09T11:19:40.313 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:41.083 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:41.363 INFO:teuthology.orchestra.run.smithi174.stdout:Total 25 MB/s | 19 MB 00:00 2019-12-09T11:19:41.364 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T11:19:41.364 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:41.364 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:41.365 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:41.365 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:41.646 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T11:19:41.774 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T11:19:41.924 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T11:19:42.210 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-12-09T11:19:42.601 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T11:19:42.679 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-12-09T11:19:42.701 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T11:19:43.203 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T11:19:43.204 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:43.204 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:43.204 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:43.204 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:43.204 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:43.205 INFO:teuthology.orchestra.run.smithi106.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T11:19:43.205 INFO:teuthology.orchestra.run.smithi106.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T11:19:43.205 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:43.205 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:43.322 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr-rook 2019-12-09T11:19:43.577 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:43.618 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:43.619 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:43.619 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:43.619 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:43.620 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:44.260 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:44.442 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-12-09T11:19:44.586 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:44.586 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:44.586 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:44.587 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:44.617 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:44.618 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T11:19:44.620 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:44.621 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:44.621 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:44.623 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:44.623 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:44.624 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T11:19:44.624 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T11:19:44.624 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T11:19:44.625 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:44.625 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:44.626 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:44.627 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T11:19:44.628 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T11:19:44.628 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:44.629 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T11:19:44.629 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T11:19:44.629 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T11:19:44.630 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T11:19:44.630 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T11:19:44.630 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:44.630 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T11:19:44.993 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:45.215 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:45.216 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:45.217 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:45.217 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:45.217 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:19:45.217 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:45.217 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T11:19:45.218 INFO:teuthology.orchestra.run.smithi106.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T11:19:45.219 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:45.220 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:45.220 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:45.220 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T11:19:45.220 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:45.221 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 1.6 M 2019-12-09T11:19:45.221 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 24 M 2019-12-09T11:19:45.221 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:46.003 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:46.003 INFO:teuthology.orchestra.run.smithi106.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-12-09T11:19:46.017 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:46.028 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:46.096 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:46.096 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:46.141 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-12-09T11:19:46.489 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 5/5 2019-12-09T11:19:46.609 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-12-09T11:19:46.700 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T11:19:46.709 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7900.g1351c81.el7.n 2/5 2019-12-09T11:19:46.818 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 3/5 2019-12-09T11:19:46.951 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-12-09T11:19:47.068 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:47.462 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:19:47.463 INFO:teuthology.orchestra.run.smithi174.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-12-09T11:19:47.463 INFO:teuthology.orchestra.run.smithi174.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-12-09T11:19:47.463 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:47.463 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:47.593 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-rook 2019-12-09T11:19:47.655 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T11:19:47.806 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:47.847 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:47.848 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:47.849 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:47.849 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:47.849 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:47.935 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T11:19:48.266 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T11:19:48.498 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:48.591 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T11:19:48.828 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:48.829 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:48.829 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:48.830 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:48.861 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:48.861 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-12-09T11:19:48.863 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:48.864 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:48.865 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:48.866 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-12-09T11:19:48.867 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:48.867 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-12-09T11:19:48.867 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-12-09T11:19:48.868 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-12-09T11:19:48.868 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:48.869 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:48.870 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-12-09T11:19:48.871 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-12-09T11:19:48.871 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-12-09T11:19:48.872 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:48.872 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-12-09T11:19:48.872 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-12-09T11:19:48.873 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-12-09T11:19:48.874 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-12-09T11:19:48.874 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-12-09T11:19:48.874 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:48.874 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-12-09T11:19:48.907 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T11:19:49.239 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:49.253 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T11:19:49.464 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:49.465 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:49.466 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:49.466 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:49.466 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 23 k 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-12-09T11:19:49.467 INFO:teuthology.orchestra.run.smithi174.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-12-09T11:19:49.468 INFO:teuthology.orchestra.run.smithi174.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+10 Dependent packages) 2019-12-09T11:19:49.469 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:49.470 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 1.6 M 2019-12-09T11:19:49.470 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 24 M 2019-12-09T11:19:49.470 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:50.254 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:50.255 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-12-09T11:19:50.271 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:19:50.285 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:19:50.357 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:19:50.357 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:19:50.541 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T11:19:50.932 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T11:19:51.127 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 1/11 2019-12-09T11:19:51.180 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T11:19:51.289 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T11:19:51.396 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T11:19:51.505 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T11:19:51.522 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 2/11 2019-12-09T11:19:51.614 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T11:19:51.723 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T11:19:51.863 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 3/11 2019-12-09T11:19:51.873 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T11:19:51.999 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T11:19:52.141 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T11:19:52.160 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 4/11 2019-12-09T11:19:52.292 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T11:19:52.393 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T11:19:52.567 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 5/11 2019-12-09T11:19:52.869 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T11:19:52.869 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:52.869 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:52.869 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:52.869 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T11:19:52.870 INFO:teuthology.orchestra.run.smithi106.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:52.871 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:52.974 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 6/11 2019-12-09T11:19:52.986 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-mgr-ssh 2019-12-09T11:19:53.249 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:53.294 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:53.295 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:53.296 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:53.296 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:53.296 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:53.481 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-12-09T11:19:53.819 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 8/11 2019-12-09T11:19:53.934 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:54.270 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:54.271 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:54.271 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:54.273 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:54.303 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:54.303 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T11:19:54.304 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T11:19:54.305 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:54.305 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T11:19:54.669 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:54.892 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:54.893 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:54.893 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:54.893 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:54.894 INFO:teuthology.orchestra.run.smithi106.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:54.895 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 316 k 2019-12-09T11:19:54.896 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 1.1 M 2019-12-09T11:19:54.896 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:19:55.110 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-12-09T11:19:55.422 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:19:55.423 INFO:teuthology.orchestra.run.smithi106.stdout:Total 598 kB/s | 316 kB 00:00 2019-12-09T11:19:55.426 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:19:55.435 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:19:55.519 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:19:55.520 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:19:55.577 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-12-09T11:19:55.888 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 11/11 2019-12-09T11:19:56.038 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-12-09T11:19:56.147 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 2/11 2019-12-09T11:19:56.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T11:19:56.272 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-12-09T11:19:56.347 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-12-09T11:19:56.431 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 5/11 2019-12-09T11:19:56.519 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T11:19:56.557 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-12-09T11:19:56.649 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-12-09T11:19:56.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-12-09T11:19:56.812 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T11:19:56.867 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-12-09T11:19:56.941 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T11:19:56.967 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7900.g1351c81.el7.noarch 10/11 2019-12-09T11:19:57.083 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T11:19:57.519 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 11/11 2019-12-09T11:19:57.519 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:57.519 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:19:57.520 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-12-09T11:19:57.521 INFO:teuthology.orchestra.run.smithi174.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-12-09T11:19:57.522 INFO:teuthology.orchestra.run.smithi174.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-12-09T11:19:57.523 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:57.523 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:19:57.652 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T11:19:57.652 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:57.652 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:19:57.652 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:19:57.653 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:57.653 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:19:57.653 INFO:teuthology.orchestra.run.smithi106.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T11:19:57.653 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:57.653 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:19:57.688 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-mgr-ssh 2019-12-09T11:19:57.773 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-fuse 2019-12-09T11:19:57.922 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:57.962 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:57.962 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:19:57.963 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:57.963 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:57.964 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:58.008 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:19:58.049 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:19:58.050 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:19:58.051 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:19:58.051 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:19:58.051 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:19:58.551 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:58.696 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:19:58.879 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:19:58.880 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:58.880 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:58.882 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 2019-12-09T11:19:58.913 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:58.913 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-12-09T11:19:58.913 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-12-09T11:19:58.914 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:19:58.915 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-12-09T11:19:59.027 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:19:59.028 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:59.028 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:19:59.038 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:59.070 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:19:59.071 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:59.071 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T11:19:59.071 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:19:59.073 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:19:59.074 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:19:59.075 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:19:59.075 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:19:59.075 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T11:19:59.081 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T11:19:59.081 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T11:19:59.283 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:59.443 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:19:59.510 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:59.511 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:19:59.511 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-ssh noarch 2:15.0.0-7900.g1351c81.el7 ceph-noarch 33 k 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:19:59.512 INFO:teuthology.orchestra.run.smithi174.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+2 Dependent packages) 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:19:59.513 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 316 k 2019-12-09T11:19:59.514 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 1.1 M 2019-12-09T11:19:59.514 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:19:59.666 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:59.666 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:19:59.667 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:59.667 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:59.667 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:19:59.667 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T11:19:59.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:19:59.669 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 10 M 2019-12-09T11:19:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 49 M 2019-12-09T11:19:59.670 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:20:00.042 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:20:00.043 INFO:teuthology.orchestra.run.smithi174.stdout:Total 596 kB/s | 316 kB 00:00 2019-12-09T11:20:00.046 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:20:00.055 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:20:00.134 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:20:00.135 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:20:00.348 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:20:00.348 INFO:teuthology.orchestra.run.smithi106.stdout:Total 15 MB/s | 10 MB 00:00 2019-12-09T11:20:00.364 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:20:00.387 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:20:00.518 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:20:00.518 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:20:00.971 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-12-09T11:20:01.286 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-12-09T11:20:01.596 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T11:20:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-12-09T11:20:01.857 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-12-09T11:20:02.375 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7900.g1351c81.el7.noarch 3/3 2019-12-09T11:20:02.376 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:02.376 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:20:02.376 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:02.376 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:02.376 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:20:02.377 INFO:teuthology.orchestra.run.smithi174.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-12-09T11:20:02.377 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:02.377 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:20:02.505 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-fuse 2019-12-09T11:20:02.737 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:02.775 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:02.776 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:02.777 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:02.902 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T11:20:03.425 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:03.430 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T11:20:03.753 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:20:03.754 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:03.754 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:03.764 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:20:03.797 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python3 for package: 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:20:03.797 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:03.798 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T11:20:03.798 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:20:03.800 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:20:03.800 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:20:03.802 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-12-09T11:20:03.802 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:03.803 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-12-09T11:20:03.808 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-12-09T11:20:03.808 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-12-09T11:20:04.169 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:04.353 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T11:20:04.396 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:04.397 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:20:04.397 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:20:04.398 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 566 k 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-12-09T11:20:04.399 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+4 Dependent packages) 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 10 M 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 49 M 2019-12-09T11:20:04.400 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:20:04.786 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T11:20:05.031 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T11:20:05.195 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T11:20:05.321 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T11:20:05.446 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T11:20:05.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T11:20:05.600 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:20:05.601 INFO:teuthology.orchestra.run.smithi174.stdout:Total 8.3 MB/s | 10 MB 00:01 2019-12-09T11:20:05.623 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:20:05.651 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:20:05.791 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:20:05.792 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:20:06.108 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T11:20:06.108 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:06.108 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:20:06.108 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:06.108 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:06.109 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:20:06.109 INFO:teuthology.orchestra.run.smithi106.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T11:20:06.109 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T11:20:06.109 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:06.109 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:20:06.227 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install libcephfs2 2019-12-09T11:20:06.507 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:06.552 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:06.553 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:06.553 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:06.553 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:06.554 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:07.202 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:07.534 INFO:teuthology.orchestra.run.smithi106.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:07.535 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do 2019-12-09T11:20:07.595 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install libcephfs-devel 2019-12-09T11:20:07.808 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:07.845 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:07.845 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:08.070 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-12-09T11:20:08.433 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:08.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-12-09T11:20:08.756 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:20:08.756 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:20:08.756 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:08.757 INFO:teuthology.orchestra.run.smithi106.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:20:08.787 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:20:08.788 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:09.160 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:09.384 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:09.385 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:20:09.385 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:09.385 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout:Installing for dependencies: 2019-12-09T11:20:09.386 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 170 k 2019-12-09T11:20:09.387 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 679 k 2019-12-09T11:20:09.388 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:20:09.499 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-12-09T11:20:09.753 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:20:09.754 INFO:teuthology.orchestra.run.smithi106.stdout:Total 462 kB/s | 170 kB 00:00 2019-12-09T11:20:09.756 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:20:09.768 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:20:09.771 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:20:09.772 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:20:10.021 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-12-09T11:20:10.334 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T11:20:10.344 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 5/5 2019-12-09T11:20:10.469 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:ceph-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/5 2019-12-09T11:20:10.563 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T11:20:10.619 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 2/5 2019-12-09T11:20:10.652 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T11:20:10.719 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-12-09T11:20:10.861 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-12-09T11:20:11.111 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:11.112 INFO:teuthology.orchestra.run.smithi106.stdout:Dependency Installed: 2019-12-09T11:20:11.113 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:11.113 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:11.113 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:20:11.245 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install librados2 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:11.255 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:20:11.256 INFO:teuthology.orchestra.run.smithi174.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-12-09T11:20:11.256 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-12-09T11:20:11.256 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:11.256 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:20:11.371 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs2 2019-12-09T11:20:11.459 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:11.497 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:11.498 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:11.499 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:11.499 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:11.499 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:11.637 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:11.682 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:11.683 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:11.683 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:11.683 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:11.684 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:12.126 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:12.326 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:12.452 INFO:teuthology.orchestra.run.smithi106.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:12.453 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do 2019-12-09T11:20:12.514 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install librbd1 2019-12-09T11:20:12.670 INFO:teuthology.orchestra.run.smithi174.stdout:Package 2:libcephfs2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:12.670 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do 2019-12-09T11:20:12.732 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install libcephfs-devel 2019-12-09T11:20:12.762 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:12.808 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:12.809 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:12.810 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:12.810 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:12.810 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:12.935 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:12.970 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:12.971 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:12.971 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:12.972 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:12.973 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:13.454 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:13.561 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:13.792 INFO:teuthology.orchestra.run.smithi106.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:13.792 INFO:teuthology.orchestra.run.smithi106.stdout:Nothing to do 2019-12-09T11:20:13.852 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install python-ceph 2019-12-09T11:20:13.890 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:20:13.890 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:13.890 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:13.891 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7900.g1351c81.el7 for package: 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2019-12-09T11:20:13.922 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:13.922 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:14.064 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:14.107 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:14.108 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:14.108 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:14.108 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:14.109 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:14.294 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:14.530 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:14.530 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:20:14.531 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:14.531 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:14.531 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:20:14.531 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 17 k 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:15.0.0-7900.g1351c81.el7 ceph 153 k 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:14.532 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package (+1 Dependent package) 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 170 k 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 679 k 2019-12-09T11:20:14.533 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:20:14.752 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:14.870 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:20:14.871 INFO:teuthology.orchestra.run.smithi174.stdout:Total 501 kB/s | 170 kB 00:00 2019-12-09T11:20:14.872 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:20:14.881 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:20:14.882 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:20:14.883 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:20:15.084 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:20:15.084 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:20:15.084 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:15.452 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:15.516 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T11:20:15.635 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:15.635 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:20:15.636 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:15.636 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:15.636 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository 2019-12-09T11:20:15.636 INFO:teuthology.orchestra.run.smithi106.stdout: Size 2019-12-09T11:20:15.636 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2019-12-09T11:20:15.637 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:15.638 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 2.2 k 2019-12-09T11:20:15.638 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 0 2019-12-09T11:20:15.638 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:20:15.865 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T11:20:15.928 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:20:15.929 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:20:15.931 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:20:15.931 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:20:15.993 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:librados-devel-15.0.0-7900.g1351c81.el7.x86_64 1/2 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:libcephfs-devel-15.0.0-7900.g1351c81.el7.x86_64 2/2 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:16.452 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:20:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:16.453 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:16.453 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:20:16.474 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:16.569 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install librados2 2019-12-09T11:20:16.800 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:16.840 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:16.841 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:16.842 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:16.842 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:16.842 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:16.867 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:16.868 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:16.868 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:20:16.868 INFO:teuthology.orchestra.run.smithi106.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:16.868 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:16.868 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:20:16.978 INFO:teuthology.orchestra.run.smithi106:> sudo yum -y install rbd-fuse 2019-12-09T11:20:17.238 INFO:teuthology.orchestra.run.smithi106.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:17.284 INFO:teuthology.orchestra.run.smithi106.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:17.285 INFO:teuthology.orchestra.run.smithi106.stdout: * base: mirror.ash.fastserv.com 2019-12-09T11:20:17.285 INFO:teuthology.orchestra.run.smithi106.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:17.285 INFO:teuthology.orchestra.run.smithi106.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:17.286 INFO:teuthology.orchestra.run.smithi106.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:17.491 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:17.825 INFO:teuthology.orchestra.run.smithi174.stdout:Package 2:librados2-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:17.826 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do 2019-12-09T11:20:17.887 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install librbd1 2019-12-09T11:20:17.928 INFO:teuthology.orchestra.run.smithi106.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:18.140 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:18.185 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:18.186 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:18.186 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:18.186 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:18.187 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:18.267 INFO:teuthology.orchestra.run.smithi106.stdout:Resolving Dependencies 2019-12-09T11:20:18.268 INFO:teuthology.orchestra.run.smithi106.stdout:--> Running transaction check 2019-12-09T11:20:18.268 INFO:teuthology.orchestra.run.smithi106.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:18.640 INFO:teuthology.orchestra.run.smithi106.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:18.823 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:18.823 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies Resolved 2019-12-09T11:20:18.824 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:18.824 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:18.824 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2019-12-09T11:20:18.824 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:18.824 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout:Install 1 Package 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:18.825 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 76 k 2019-12-09T11:20:18.826 INFO:teuthology.orchestra.run.smithi106.stdout:Installed size: 200 k 2019-12-09T11:20:18.826 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading packages: 2019-12-09T11:20:18.832 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:19.129 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2019-12-09T11:20:19.151 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2019-12-09T11:20:19.152 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded 2019-12-09T11:20:19.153 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2019-12-09T11:20:19.170 INFO:teuthology.orchestra.run.smithi174.stdout:Package 2:librbd1-15.0.0-7900.g1351c81.el7.x86_64 already installed and latest version 2019-12-09T11:20:19.170 INFO:teuthology.orchestra.run.smithi174.stdout:Nothing to do 2019-12-09T11:20:19.232 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install python-ceph 2019-12-09T11:20:19.397 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:19.431 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:19.432 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:19.433 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:19.433 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:19.433 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:19.787 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:20.022 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:20.296 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:20.296 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:20.296 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2019-12-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi106.stdout: 2019-12-09T11:20:20.297 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2019-12-09T11:20:20.351 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:20:20.352 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:20.352 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:20.404 DEBUG:teuthology.parallel:result is None 2019-12-09T11:20:20.724 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:20.908 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:20.909 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:20:20.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout: Size 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:20:20.910 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-compat x86_64 2:15.0.0-7900.g1351c81.el7 ceph 2.2 k 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 2.2 k 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 0 2019-12-09T11:20:20.911 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:20:21.315 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:20:21.316 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:20:21.317 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:20:21.317 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:20:21.889 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:22.422 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:python-ceph-compat-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:22.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:22.423 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:20:22.423 INFO:teuthology.orchestra.run.smithi174.stdout: python-ceph-compat.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:22.423 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:22.423 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:20:22.543 INFO:teuthology.orchestra.run.smithi174:> sudo yum -y install rbd-fuse 2019-12-09T11:20:22.793 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:20:22.837 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:20:22.838 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:20:22.838 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:20:22.839 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:20:22.839 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:20:23.486 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:20:23.823 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:20:23.824 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:20:23.824 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 will be installed 2019-12-09T11:20:24.200 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:20:24.386 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:24.386 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:20:24.387 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:24.387 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:24.387 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:20:24.388 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:24.388 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:20:24.388 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:15.0.0-7900.g1351c81.el7 ceph 76 k 2019-12-09T11:20:24.388 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:24.388 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:20:24.389 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:20:24.389 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T11:20:24.389 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:24.389 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 76 k 2019-12-09T11:20:24.390 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 200 k 2019-12-09T11:20:24.390 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:20:24.809 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:20:24.831 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:20:24.833 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:20:24.834 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:20:25.492 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:25.944 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : 2:rbd-fuse-15.0.0-7900.g1351c81.el7.x86_64 1/1 2019-12-09T11:20:25.944 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:25.944 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:20:25.945 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse.x86_64 2:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:25.945 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:20:25.945 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:20:26.067 DEBUG:teuthology.parallel:result is None 2019-12-09T11:20:26.067 INFO:teuthology.orchestra.run.smithi106:> sudo lsb_release -is 2019-12-09T11:20:26.219 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS 2019-12-09T11:20:26.221 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:20:26.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:20:26.381 INFO:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T11:20:26.518 INFO:teuthology.orchestra.run.smithi106.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:26.519 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T11:20:26.519 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T11:20:26.519 INFO:teuthology.orchestra.run.smithi174:> sudo lsb_release -is 2019-12-09T11:20:26.672 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS 2019-12-09T11:20:26.674 DEBUG:teuthology.misc:System to be installed: CentOS 2019-12-09T11:20:26.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:20:26.838 INFO:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-12-09T11:20:26.975 INFO:teuthology.orchestra.run.smithi174.stdout:15.0.0-7900.g1351c81.el7 2019-12-09T11:20:26.975 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7900.g1351c81.el7 2019-12-09T11:20:26.975 INFO:teuthology.task.install:The correct ceph version 15.0.0-7900.g1351c81 is installed. 2019-12-09T11:20:26.976 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-12-09T11:20:26.976 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T11:20:27.104 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-12-09T11:20:27.223 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-12-09T11:20:27.224 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T11:20:27.339 INFO:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T11:20:27.433 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-12-09T11:20:27.551 INFO:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-12-09T11:20:27.639 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-12-09T11:20:27.640 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T11:20:27.757 INFO:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T11:20:27.865 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-12-09T11:20:27.984 INFO:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-12-09T11:20:28.092 INFO:teuthology.run_tasks:Running task ceph... 2019-12-09T11:20:28.162 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-12-09T11:20:28.163 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2019-12-09T11:20:28.167 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 777 /var/log/ceph 2019-12-09T11:20:28.284 INFO:tasks.ceph:Disabling ceph logrotate... 2019-12-09T11:20:28.284 INFO:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T11:20:28.288 INFO:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/logrotate.d/ceph 2019-12-09T11:20:28.396 INFO:tasks.ceph:Creating extra log directories... 2019-12-09T11:20:28.396 INFO:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T11:20:28.400 INFO:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-12-09T11:20:28.511 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-12-09T11:20:28.511 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-12-09T11:20:28.512 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.8229', 'owner': 'scheduled_ofriedma@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi174.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC3DSmS5rlGuNxmSFQRAnHMrHbZg4ztgE7CYu35NfteOEMP1P0f/aPvaQri8Lkr1ts1zs2LmyOBxvXli5VP91y2tA7kTrCc6SB15TqzIQrMccq6H6uxA8g6KyBrAWTimXGoxpa4n91ZYxhdI67Fuz/j/OqRxHnOpVLedv70RWRwNKm0xL6cZLuIlBibb1zcvT5QiwE4FFug3YNBzZEKBQU15weGHYEjPpjy+lVQm3BfJ6QOfqWiiZiC84oPvCtepurT4X3Mr/wXqkcCspE00PSVLNmHMOF2tGgdKeDgwgROvKOg1juiXIqAjkjA3GV4RaIQBirkPDXSzxreGANcExnn', 'smithi106.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC4Y2hUa3UcyVVQvDv9tlcLB8dL8kTGA/vbCDskZ4q/lbHQ8OUNeybWaTIkW+Eg8mtVWcfotE9eM3CCYVs43vbgoGx7Nvl0jFLxmSc42NxzateYVE+CCB98xA+4pCwxbZ476SNaVOl/PcloPG3xwA7UIx2i0UwRT37H8OUwzGB3YuYLCp0tJZTf1k9s35T+OJTJMnCEDlghC3ayLtIIhtQkaW0QG4Xloqs6ER99D8sCmEr4lGoSJ1yxNAjTdHosHzgCeVn1MRfqfwE9nxpULaO0w9RskLkCllUUlAPgTGPHv4NlQ+5g6JMADcDhHNdpdAwaefyU2LCgEhukij4vT1H'}, 'job_id': '4584290', 'tube': 'smithi', 'suite_branch': 'wip-ofriedma-dynamic-disable', 'priority': 60, 'first_in_suite': False, 'branch': 'wip-ofriedma-dynamic-disable', 'suite': 'rgw', 'email': 'ofriedma@redhat.com', 'archive_path': '/home/teuthworker/archive/ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi/4584290', 'description': 'rgw/verify/{clusters/fixed-2.yaml frontend/beast.yaml msgr-failures/few.yaml objectstore/bluestore-bitmap.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'name': 'ofriedma-2019-12-09_08:35:41-rgw-wip-ofriedma-dynamic-disable-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'os_type': 'centos', 'overrides': {'s3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'mon osd full ratio': 0.9, 'mon osd backfillfull_ratio': 0.85, 'bluestore fsck on mount': True, 'mon osd nearfull ratio': 0.8, 'debug bluestore': 20, 'debug bluefs': 20, 'osd objectstore': 'bluestore', 'bluestore block size': 96636764160, 'debug rocksdb': 10, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95, 'bdev async discard': True}}}, 'rgw': {'frontend': 'beast', 'compression type': 'random', 'storage classes': 'LUKEWARM, FROZEN', 'ec-data-pool': True}, 'workunit': {'sha1': '1351c819e1a9518cde4c8b12d645782009028360', 'branch': 'wip-ofriedma-dynamic-disable'}, 'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'mon osd full ratio': 0.9, 'debug ms': 1, 'bluestore fsck on mount': True, 'debug osd': 25, 'debug bluestore': 20, 'debug bluefs': 20, 'mon osd backfillfull_ratio': 0.85, 'debug rocksdb': 10, 'osd shutdown pgref assert': True, 'debug journal': 20, 'bluestore allocator': 'bitmap', 'mon osd nearfull ratio': 0.8, 'osd objectstore': 'bluestore', 'bdev async discard': True, 'osd fast shutdown': False, 'bluestore block size': 96636764160, 'debug filestore': 20, 'bdev enable discard': True, 'osd failsafe full ratio': 0.95}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '1351c819e1a9518cde4c8b12d645782009028360'}}, 'admin_socket': {'branch': 'wip-ofriedma-dynamic-disable'}, 'thrashosds': {'bdev_inject_crash_probability': 0.5, 'bdev_inject_crash': 2}}} 2019-12-09T11:20:28.512 INFO:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T11:20:28.516 INFO:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-12-09T11:20:28.596 INFO:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T11:20:28.599 INFO:teuthology.orchestra.run.smithi174:> sudo install -d -m0777 -- /var/run/ceph 2019-12-09T11:20:28.791 DEBUG:teuthology.orchestra.remote:smithi106:/scratch_devs is 72B 2019-12-09T11:20:28.863 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T11:20:28.863 INFO:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T11:20:28.920 INFO:teuthology.orchestra.run.smithi106.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T11:20:28.920 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:28.920 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 103212 Links: 1 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2019-12-09 11:19:15.020364364 +0000 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2019-12-09 11:17:28.683351574 +0000 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2019-12-09 11:17:28.683351574 +0000 2019-12-09T11:20:28.921 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2019-12-09T11:20:28.938 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2019-12-09T11:20:28.938 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2019-12-09T11:20:28.938 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes (512 B) copied, 0.000319733 s, 1.6 MB/s 2019-12-09T11:20:28.941 INFO:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T11:20:28.990 INFO:teuthology.orchestra.run.smithi106.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T11:20:28.990 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:28.990 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 109678 Links: 1 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2019-12-09 11:19:15.020364364 +0000 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2019-12-09 11:17:29.111341416 +0000 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2019-12-09 11:17:29.111341416 +0000 2019-12-09T11:20:28.991 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2019-12-09T11:20:29.008 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2019-12-09T11:20:29.009 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2019-12-09T11:20:29.009 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes (512 B) copied, 0.000290426 s, 1.8 MB/s 2019-12-09T11:20:29.012 INFO:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T11:20:29.058 INFO:teuthology.orchestra.run.smithi106.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T11:20:29.059 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.059 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 106886 Links: 1 2019-12-09T11:20:29.059 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.059 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.059 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2019-12-09 11:19:15.020364364 +0000 2019-12-09T11:20:29.060 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2019-12-09 11:17:29.533331400 +0000 2019-12-09T11:20:29.060 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2019-12-09 11:17:29.533331400 +0000 2019-12-09T11:20:29.060 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2019-12-09T11:20:29.075 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2019-12-09T11:20:29.076 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2019-12-09T11:20:29.076 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes (512 B) copied, 0.000286015 s, 1.8 MB/s 2019-12-09T11:20:29.079 INFO:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T11:20:29.125 INFO:teuthology.orchestra.run.smithi106.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T11:20:29.125 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.125 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 106916 Links: 1 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2019-12-09 11:19:15.020364364 +0000 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2019-12-09 11:17:30.018319890 +0000 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2019-12-09 11:17:30.018319890 +0000 2019-12-09T11:20:29.126 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: - 2019-12-09T11:20:29.142 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2019-12-09T11:20:29.142 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2019-12-09T11:20:29.142 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes (512 B) copied, 0.000346735 s, 1.5 MB/s 2019-12-09T11:20:29.145 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T11:20:29.145 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T11:20:29.145 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T11:20:29.146 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:29.241 DEBUG:teuthology.orchestra.remote:smithi174:/scratch_devs is 72B 2019-12-09T11:20:29.323 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T11:20:29.323 INFO:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-12-09T11:20:29.402 INFO:teuthology.orchestra.run.smithi174.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 92851 Links: 1 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2019-12-09 11:19:18.883871434 +0000 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2019-12-09 11:17:33.971183974 +0000 2019-12-09T11:20:29.403 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2019-12-09 11:17:33.971183974 +0000 2019-12-09T11:20:29.404 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2019-12-09T11:20:29.431 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2019-12-09T11:20:29.431 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2019-12-09T11:20:29.432 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes (512 B) copied, 0.000396504 s, 1.3 MB/s 2019-12-09T11:20:29.437 INFO:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-12-09T11:20:29.513 INFO:teuthology.orchestra.run.smithi174.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 107091 Links: 1 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2019-12-09 11:19:18.883871434 +0000 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2019-12-09 11:17:34.432178165 +0000 2019-12-09T11:20:29.514 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2019-12-09 11:17:34.432178165 +0000 2019-12-09T11:20:29.515 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2019-12-09T11:20:29.540 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2019-12-09T11:20:29.540 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2019-12-09T11:20:29.540 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes (512 B) copied, 0.000410678 s, 1.2 MB/s 2019-12-09T11:20:29.545 INFO:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-12-09T11:20:29.611 INFO:teuthology.orchestra.run.smithi174.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-12-09T11:20:29.612 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.612 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 100206 Links: 1 2019-12-09T11:20:29.612 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.612 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.613 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2019-12-09 11:19:18.883871434 +0000 2019-12-09T11:20:29.613 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2019-12-09 11:17:34.947171677 +0000 2019-12-09T11:20:29.613 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2019-12-09 11:17:34.947171677 +0000 2019-12-09T11:20:29.613 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2019-12-09T11:20:29.635 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2019-12-09T11:20:29.635 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2019-12-09T11:20:29.635 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes (512 B) copied, 0.000357076 s, 1.4 MB/s 2019-12-09T11:20:29.641 INFO:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-12-09T11:20:29.702 INFO:teuthology.orchestra.run.smithi174.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-12-09T11:20:29.702 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-12-09T11:20:29.702 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 99279 Links: 1 2019-12-09T11:20:29.702 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-12-09T11:20:29.702 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2019-12-09T11:20:29.703 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2019-12-09 11:19:18.883871434 +0000 2019-12-09T11:20:29.703 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2019-12-09 11:17:35.401165957 +0000 2019-12-09T11:20:29.703 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2019-12-09 11:17:35.401165957 +0000 2019-12-09T11:20:29.703 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: - 2019-12-09T11:20:29.719 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2019-12-09T11:20:29.720 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2019-12-09T11:20:29.720 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes (512 B) copied, 0.000306126 s, 1.7 MB/s 2019-12-09T11:20:29.723 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-12-09T11:20:29.723 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-12-09T11:20:29.723 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-12-09T11:20:29.723 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:29.723 INFO:tasks.ceph:Generating config... 2019-12-09T11:20:29.724 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-12-09T11:20:29.725 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-12-09T11:20:29.725 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-12-09T11:20:29.725 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-12-09T11:20:29.725 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-12-09T11:20:29.725 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2019-12-09T11:20:29.726 INFO:tasks.ceph:[osd] debug ms = 1 2019-12-09T11:20:29.727 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2019-12-09T11:20:29.727 INFO:tasks.ceph:[osd] debug osd = 25 2019-12-09T11:20:29.727 INFO:tasks.ceph:[osd] debug bluestore = 20 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] debug bluefs = 20 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] debug rocksdb = 10 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] debug journal = 20 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2019-12-09T11:20:29.728 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] bdev async discard = True 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] osd fast shutdown = False 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] debug filestore = 20 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] bdev enable discard = True 2019-12-09T11:20:29.729 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2019-12-09T11:20:29.730 INFO:tasks.ceph:[mon] debug mon = 20 2019-12-09T11:20:29.730 INFO:tasks.ceph:[mon] debug paxos = 20 2019-12-09T11:20:29.730 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-12-09T11:20:29.730 INFO:tasks.ceph:[mon] debug ms = 1 2019-12-09T11:20:29.730 INFO:tasks.ceph:[client] debug rgw = 20 2019-12-09T11:20:29.730 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-12-09T11:20:29.731 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-12-09T11:20:29.731 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-12-09T11:20:29.731 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-12-09T11:20:29.731 INFO:tasks.ceph:Setting up mon.a... 2019-12-09T11:20:29.731 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-12-09T11:20:29.862 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.keyring 2019-12-09T11:20:29.867 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-12-09T11:20:30.000 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T11:20:30.100 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.174'), ('mon.c', '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]'), ('mon.b', '172.21.15.106')] 2019-12-09T11:20:30.100 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.174 --addv c '[v2:172.21.15.174:3301,v1:172.21.15.174:6790]' --add b 172.21.15.106 --print /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:30.206 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:30.206 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: generated fsid f7df3c5f-47d5-4d9c-8d28-009d70e50517 2019-12-09T11:20:30.206 INFO:teuthology.orchestra.run.smithi174.stdout:epoch 0 2019-12-09T11:20:30.206 INFO:teuthology.orchestra.run.smithi174.stdout:fsid f7df3c5f-47d5-4d9c-8d28-009d70e50517 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:last_changed 2019-12-09T11:20:30.207388+0000 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:created 2019-12-09T11:20:30.207388+0000 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:min_mon_release 0 (unknown) 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:0: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.a 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:1: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2019-12-09T11:20:30.207 INFO:teuthology.orchestra.run.smithi174.stdout:2: [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] mon.c 2019-12-09T11:20:30.208 INFO:teuthology.orchestra.run.smithi174.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-12-09T11:20:30.209 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f7df3c5f-47d5-4d9c-8d28-009d70e50517... 2019-12-09T11:20:30.210 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T11:20:30.214 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-12-09T11:20:30.426 INFO:tasks.ceph:Creating admin key on mon.a... 2019-12-09T11:20:30.426 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-12-09T11:20:30.548 INFO:tasks.ceph:Copying monmap to all nodes... 2019-12-09T11:20:30.616 DEBUG:teuthology.orchestra.remote:smithi174:/etc/ceph/ceph.keyring is 206B 2019-12-09T11:20:30.766 DEBUG:teuthology.orchestra.remote:smithi174:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-12-09T11:20:30.834 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:20:30.835 INFO:teuthology.orchestra.run.smithi106:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T11:20:30.989 INFO:teuthology.orchestra.run.smithi106:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:31.060 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:20:31.060 INFO:teuthology.orchestra.run.smithi174:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-12-09T11:20:31.212 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:31.292 INFO:tasks.ceph:Setting up mon nodes... 2019-12-09T11:20:31.292 INFO:tasks.ceph:Setting up mgr nodes... 2019-12-09T11:20:31.293 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T11:20:31.521 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-12-09T11:20:31.525 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T11:20:31.693 INFO:teuthology.orchestra.run.smithi174.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-12-09T11:20:31.697 INFO:tasks.ceph:Setting up mds nodes... 2019-12-09T11:20:31.697 INFO:tasks.ceph_client:Setting up client nodes... 2019-12-09T11:20:31.697 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-12-09T11:20:31.843 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-12-09T11:20:31.878 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-12-09T11:20:32.022 INFO:teuthology.orchestra.run.smithi174.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-12-09T11:20:32.055 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-12-09T11:20:32.055 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi174.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}} 2019-12-09T11:20:32.055 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-12-09T11:20:32.174 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:32.174 INFO:tasks.ceph:{} 2019-12-09T11:20:32.174 INFO:tasks.ceph:osd.4 2019-12-09T11:20:32.174 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:20:32.174 INFO:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T11:20:32.782 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:32.783 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:32.784 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:32.784 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:32.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2019-12-09T11:20:32.787 INFO:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-12-09T11:20:32.964 INFO:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-12-09T11:20:33.076 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-12-09T11:20:33.160 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:33.160 INFO:tasks.ceph:{} 2019-12-09T11:20:33.160 INFO:tasks.ceph:osd.5 2019-12-09T11:20:33.161 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:20:33.161 INFO:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T11:20:33.709 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:33.710 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:33.710 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:33.710 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:33.710 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:33.710 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:33.711 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:33.711 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:33.711 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:33.712 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2019-12-09T11:20:33.712 INFO:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-12-09T11:20:33.862 INFO:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-12-09T11:20:33.971 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-12-09T11:20:34.073 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:34.073 INFO:tasks.ceph:{} 2019-12-09T11:20:34.073 INFO:tasks.ceph:osd.6 2019-12-09T11:20:34.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:20:34.074 INFO:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:34.657 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:34.658 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:34.658 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:34.658 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:34.658 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:34.658 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:34.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2019-12-09T11:20:34.660 INFO:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-12-09T11:20:34.841 INFO:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-12-09T11:20:34.959 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-12-09T11:20:35.048 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:35.049 INFO:tasks.ceph:{} 2019-12-09T11:20:35.049 INFO:tasks.ceph:osd.7 2019-12-09T11:20:35.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:20:35.049 INFO:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T11:20:35.578 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:35.579 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:35.580 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:35.580 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:35.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2019-12-09T11:20:35.581 INFO:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-12-09T11:20:35.727 INFO:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-12-09T11:20:35.843 INFO:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:35.990 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:35.991+0000 7f2b47e80b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-12-09T11:20:35.991 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:35.993+0000 7f2b47e80b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-12-09T11:20:35.992 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:35.993+0000 7f2b47e80b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T11:20:35.992 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:35.993+0000 7f2b47e80b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T11:20:35.992 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:35.993+0000 7f2b47e80b40 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2019-12-09T11:20:35.998 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:36.000+0000 7f2b47e80b40 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2019-12-09T11:20:38.307 INFO:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-12-09T11:20:38.426 INFO:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:38.573 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.575+0000 7f992be5bb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-12-09T11:20:38.574 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.576+0000 7f992be5bb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-12-09T11:20:38.575 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.576+0000 7f992be5bb40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T11:20:38.575 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.576+0000 7f992be5bb40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T11:20:38.575 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.576+0000 7f992be5bb40 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2019-12-09T11:20:38.575 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:38.577+0000 7f992be5bb40 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2019-12-09T11:20:40.884 INFO:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-12-09T11:20:41.000 INFO:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:41.135 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.137+0000 7fd366df4b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-12-09T11:20:41.136 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.137+0000 7fd366df4b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-12-09T11:20:41.136 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.138+0000 7fd366df4b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T11:20:41.137 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.138+0000 7fd366df4b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T11:20:41.137 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.138+0000 7fd366df4b40 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2019-12-09T11:20:41.137 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:41.139+0000 7fd366df4b40 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2019-12-09T11:20:43.449 INFO:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-12-09T11:20:43.568 INFO:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:43.704 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.706+0000 7f1f5c3a1b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-12-09T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.706+0000 7f1f5c3a1b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-12-09T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.706+0000 7f1f5c3a1b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.706+0000 7f1f5c3a1b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T11:20:43.705 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.706+0000 7f1f5c3a1b40 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2019-12-09T11:20:43.706 INFO:teuthology.orchestra.run.smithi106.stderr:2019-12-09T11:20:43.708+0000 7f1f5c3a1b40 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2019-12-09T11:20:46.017 INFO:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-12-09T11:20:46.133 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-12-09T11:20:46.255 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:46.255 INFO:tasks.ceph:{} 2019-12-09T11:20:46.255 INFO:tasks.ceph:osd.0 2019-12-09T11:20:46.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:20:46.256 INFO:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-12-09T11:20:46.838 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:46.839 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:46.840 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:46.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2019-12-09T11:20:46.841 INFO:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-12-09T11:20:47.019 INFO:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-12-09T11:20:47.136 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-12-09T11:20:47.229 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:47.229 INFO:tasks.ceph:{} 2019-12-09T11:20:47.229 INFO:tasks.ceph:osd.1 2019-12-09T11:20:47.229 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:20:47.230 INFO:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-12-09T11:20:47.755 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:47.755 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:47.756 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:47.757 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:47.758 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2019-12-09T11:20:47.758 INFO:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-12-09T11:20:47.906 INFO:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-12-09T11:20:47.999 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-12-09T11:20:48.077 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:48.077 INFO:tasks.ceph:{} 2019-12-09T11:20:48.077 INFO:tasks.ceph:osd.2 2019-12-09T11:20:48.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:20:48.077 INFO:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-12-09T11:20:48.638 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:48.638 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:48.638 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:48.638 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:48.639 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:48.639 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:48.639 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:48.639 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:48.639 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:48.641 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2019-12-09T11:20:48.641 INFO:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-12-09T11:20:48.813 INFO:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-12-09T11:20:48.916 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-12-09T11:20:49.002 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-12-09T11:20:49.003 INFO:tasks.ceph:{} 2019-12-09T11:20:49.003 INFO:tasks.ceph:osd.3 2019-12-09T11:20:49.003 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:20:49.003 INFO:teuthology.orchestra.run.smithi174:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-12-09T11:20:49.546 INFO:teuthology.orchestra.run.smithi174.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-12-09T11:20:49.546 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 attr=2, projid32bit=1 2019-12-09T11:20:49.546 INFO:teuthology.orchestra.run.smithi174.stdout: = crc=1 finobt=0, sparse=0 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout: = sunit=0 swidth=0 blks 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-12-09T11:20:49.547 INFO:teuthology.orchestra.run.smithi174.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-12-09T11:20:49.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi174.front.sepia.ceph.com -o noatime 2019-12-09T11:20:49.549 INFO:teuthology.orchestra.run.smithi174:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-12-09T11:20:49.655 INFO:teuthology.orchestra.run.smithi174:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-12-09T11:20:49.732 INFO:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:49.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.838+0000 7f0fe52f5b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-12-09T11:20:49.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.838+0000 7f0fe52f5b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-12-09T11:20:49.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.838+0000 7f0fe52f5b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T11:20:49.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.838+0000 7f0fe52f5b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T11:20:49.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.838+0000 7f0fe52f5b40 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2019-12-09T11:20:49.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:49.839+0000 7f0fe52f5b40 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2019-12-09T11:20:52.149 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-12-09T11:20:52.264 INFO:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:52.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.414+0000 7f60be89ab40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-12-09T11:20:52.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.415+0000 7f60be89ab40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-12-09T11:20:52.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.415+0000 7f60be89ab40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T11:20:52.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.415+0000 7f60be89ab40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T11:20:52.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.415+0000 7f60be89ab40 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2019-12-09T11:20:52.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:52.416+0000 7f60be89ab40 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2019-12-09T11:20:54.723 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-12-09T11:20:54.833 INFO:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:54.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.967+0000 7efd4ddffb40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-12-09T11:20:54.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.967+0000 7efd4ddffb40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-12-09T11:20:54.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.967+0000 7efd4ddffb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T11:20:54.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.967+0000 7efd4ddffb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T11:20:54.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.967+0000 7efd4ddffb40 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2019-12-09T11:20:54.969 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:54.969+0000 7efd4ddffb40 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2019-12-09T11:20:57.275 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-12-09T11:20:57.391 INFO:teuthology.orchestra.run.smithi174:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:20:57.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.533+0000 7f76a6fd9b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-12-09T11:20:57.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.533+0000 7f76a6fd9b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-12-09T11:20:57.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.533+0000 7f76a6fd9b40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T11:20:57.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.533+0000 7f76a6fd9b40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T11:20:57.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.533+0000 7f76a6fd9b40 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2019-12-09T11:20:57.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:20:57.535+0000 7f76a6fd9b40 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2019-12-09T11:20:59.841 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-12-09T11:20:59.957 INFO:tasks.ceph:Reading keys from all nodes... 2019-12-09T11:20:59.957 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:00.049 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.0aAP3tdqr8 2019-12-09T11:21:00.050 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.0aAP3tdqr8 2019-12-09T11:21:00.153 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.0aAP3tdqr8 2019-12-09T11:21:00.312 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.0aAP3tdqr8 is 56B 2019-12-09T11:21:00.379 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.0aAP3tdqr8 2019-12-09T11:21:00.438 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:00.493 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.KmZ1j4UF3c 2019-12-09T11:21:00.493 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.KmZ1j4UF3c 2019-12-09T11:21:00.562 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.KmZ1j4UF3c 2019-12-09T11:21:00.674 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.KmZ1j4UF3c is 56B 2019-12-09T11:21:00.726 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.KmZ1j4UF3c 2019-12-09T11:21:00.776 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:00.882 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.OP0lGEa1Vd 2019-12-09T11:21:00.882 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.OP0lGEa1Vd 2019-12-09T11:21:00.961 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.OP0lGEa1Vd 2019-12-09T11:21:01.100 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.OP0lGEa1Vd is 56B 2019-12-09T11:21:01.162 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.OP0lGEa1Vd 2019-12-09T11:21:01.222 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:01.274 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.eWhQCw2oul 2019-12-09T11:21:01.275 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.eWhQCw2oul 2019-12-09T11:21:01.342 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.eWhQCw2oul 2019-12-09T11:21:01.459 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.eWhQCw2oul is 56B 2019-12-09T11:21:01.516 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.eWhQCw2oul 2019-12-09T11:21:01.565 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:01.686 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.u0jJVWib6a 2019-12-09T11:21:01.686 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.u0jJVWib6a 2019-12-09T11:21:01.798 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.u0jJVWib6a 2019-12-09T11:21:01.998 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.u0jJVWib6a is 56B 2019-12-09T11:21:02.072 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.u0jJVWib6a 2019-12-09T11:21:02.134 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:02.230 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.Fc13ns1aUr 2019-12-09T11:21:02.231 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.Fc13ns1aUr 2019-12-09T11:21:02.332 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.Fc13ns1aUr 2019-12-09T11:21:02.485 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.Fc13ns1aUr is 56B 2019-12-09T11:21:02.561 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.Fc13ns1aUr 2019-12-09T11:21:02.621 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:02.673 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.UJ0kEzM7nB 2019-12-09T11:21:02.674 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.UJ0kEzM7nB 2019-12-09T11:21:02.740 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.UJ0kEzM7nB 2019-12-09T11:21:02.860 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.UJ0kEzM7nB is 56B 2019-12-09T11:21:02.911 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.UJ0kEzM7nB 2019-12-09T11:21:02.958 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:03.064 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.SEmsHi6Z21 2019-12-09T11:21:03.065 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.SEmsHi6Z21 2019-12-09T11:21:03.149 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.SEmsHi6Z21 2019-12-09T11:21:03.304 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.SEmsHi6Z21 is 56B 2019-12-09T11:21:03.363 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.SEmsHi6Z21 2019-12-09T11:21:03.424 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:03.550 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.Hus7P4qsc4 2019-12-09T11:21:03.550 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.Hus7P4qsc4 2019-12-09T11:21:03.645 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.Hus7P4qsc4 2019-12-09T11:21:03.786 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.Hus7P4qsc4 is 56B 2019-12-09T11:21:03.851 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.Hus7P4qsc4 2019-12-09T11:21:03.902 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:03.950 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.6eydBk5QTu 2019-12-09T11:21:03.950 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.6eydBk5QTu 2019-12-09T11:21:04.016 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.6eydBk5QTu 2019-12-09T11:21:04.132 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.6eydBk5QTu is 56B 2019-12-09T11:21:04.186 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.6eydBk5QTu 2019-12-09T11:21:04.330 DEBUG:teuthology.orchestra.remote:smithi106:/etc/ceph/ceph.client.1.keyring is 59B 2019-12-09T11:21:04.504 DEBUG:teuthology.orchestra.remote:smithi174:/etc/ceph/ceph.client.0.keyring is 59B 2019-12-09T11:21:04.589 INFO:tasks.ceph:Adding keys to all mons... 2019-12-09T11:21:04.590 INFO:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T11:21:04.594 INFO:teuthology.orchestra.run.smithi174:> sudo tee -a /etc/ceph/ceph.keyring 2019-12-09T11:21:04.705 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2019-12-09T11:21:04.705 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQD/Le5dtKI6HxAAKToEXGMcwlF2r72MiFXqRA== 2019-12-09T11:21:04.705 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2019-12-09T11:21:04.705 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQADLu5dXHIqOxAA9NVFGsxJ4lB5vpbJGjbGzw== 2019-12-09T11:21:04.705 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2019-12-09T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAGLu5dUL5dIhAACbLBj3++q8btckZ8fH8NVg== 2019-12-09T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2019-12-09T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAJLu5dZzc/CBAAiM6UL+FYNSYmkkApACHbKg== 2019-12-09T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2019-12-09T11:21:04.706 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQALLu5d8xclKhAA88AYQFHg+NrqWJLK49jsZQ== 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQD/Le5dFIxhKRAAAY5vwt0b8iBO9L2/WcN/8w== 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQARLu5dmpMRMhAAh2i/n+hrMVH3S6/PWCDGiw== 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAULu5deC7VGBAAfwBjpBJfV6B8scCHNLZRZQ== 2019-12-09T11:21:04.707 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2019-12-09T11:21:04.708 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAWLu5dNsy+ORAAqwfK9NngPuxwO3dlS8GxNg== 2019-12-09T11:21:04.708 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2019-12-09T11:21:04.708 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAZLu5dMyvhHxAACiud+lvChZJmpASFc23Vqg== 2019-12-09T11:21:04.708 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2019-12-09T11:21:04.708 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQD/Le5d3HZxMhAAPYQXkvTMNunBy4BqSlZgVw== 2019-12-09T11:21:04.709 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2019-12-09T11:21:04.709 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQAALu5d9/ZkARAApYc+R2EnB7xcHovXYPfRAw== 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.x] 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/Le5dtKI6HxAAKToEXGMcwlF2r72MiFXqRA== 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.4] 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQADLu5dXHIqOxAA9NVFGsxJ4lB5vpbJGjbGzw== 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.5] 2019-12-09T11:21:04.711 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAGLu5dUL5dIhAACbLBj3++q8btckZ8fH8NVg== 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.6] 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAJLu5dZzc/CBAAiM6UL+FYNSYmkkApACHbKg== 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.7] 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQALLu5d8xclKhAA88AYQFHg+NrqWJLK49jsZQ== 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout:[mgr.y] 2019-12-09T11:21:04.712 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/Le5dFIxhKRAAAY5vwt0b8iBO9L2/WcN/8w== 2019-12-09T11:21:04.713 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.0] 2019-12-09T11:21:04.713 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQARLu5dmpMRMhAAh2i/n+hrMVH3S6/PWCDGiw== 2019-12-09T11:21:04.713 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.1] 2019-12-09T11:21:04.713 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAULu5deC7VGBAAfwBjpBJfV6B8scCHNLZRZQ== 2019-12-09T11:21:04.713 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.2] 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAWLu5dNsy+ORAAqwfK9NngPuxwO3dlS8GxNg== 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout:[osd.3] 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAZLu5dMyvhHxAACiud+lvChZJmpASFc23Vqg== 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout:[client.1] 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQD/Le5d3HZxMhAAPYQXkvTMNunBy4BqSlZgVw== 2019-12-09T11:21:04.714 INFO:teuthology.orchestra.run.smithi174.stdout:[client.0] 2019-12-09T11:21:04.715 INFO:teuthology.orchestra.run.smithi174.stdout: key = AQAALu5d9/ZkARAApYc+R2EnB7xcHovXYPfRAw== 2019-12-09T11:21:04.715 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T11:21:04.719 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T11:21:04.889 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:04.893 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.063 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.067 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.231 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.235 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.403 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.407 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.578 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T11:21:05.582 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-12-09T11:21:05.751 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.755 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.921 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:05.927 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:06.091 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:06.095 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:06.259 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:06.263 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-12-09T11:21:06.434 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T11:21:06.437 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T11:21:06.607 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T11:21:06.610 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-12-09T11:21:06.781 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-12-09T11:21:06.782 INFO:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-12-09T11:21:06.890 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T11:21:07.034 INFO:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-12-09T11:21:07.139 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-12-09T11:21:07.261 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T11:21:07.440 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-12-09T11:21:07.552 INFO:teuthology.orchestra.run.smithi174:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-12-09T11:21:07.650 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-12-09T11:21:07.792 INFO:teuthology.orchestra.run.smithi174:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-12-09T11:21:07.879 INFO:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:21:07.883 INFO:teuthology.orchestra.run.smithi174:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-12-09T11:21:07.969 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-12-09T11:21:07.969 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T11:21:07.969 INFO:tasks.ceph.mon.b:Restarting daemon 2019-12-09T11:21:07.970 INFO:teuthology.orchestra.run.smithi106:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-12-09T11:21:07.973 INFO:tasks.ceph.mon.b:Started 2019-12-09T11:21:07.973 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T11:21:07.974 INFO:tasks.ceph.mon.a:Restarting daemon 2019-12-09T11:21:07.974 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-12-09T11:21:07.978 INFO:tasks.ceph.mon.a:Started 2019-12-09T11:21:07.978 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-12-09T11:21:07.978 INFO:tasks.ceph.mon.c:Restarting daemon 2019-12-09T11:21:07.978 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-12-09T11:21:07.982 INFO:tasks.ceph.mon.c:Started 2019-12-09T11:21:07.982 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-12-09T11:21:07.982 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-12-09T11:21:07.982 INFO:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-12-09T11:21:07.986 INFO:tasks.ceph.mgr.x:Started 2019-12-09T11:21:07.987 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-12-09T11:21:07.987 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-12-09T11:21:07.991 INFO:tasks.ceph.mgr.y:Started 2019-12-09T11:21:07.991 INFO:tasks.ceph:Setting crush tunables to default 2019-12-09T11:21:07.991 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd crush tunables default 2019-12-09T11:21:11.973 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:00:03.800 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:21:12.038 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:00:03.866 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:21:12.170 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:00:03.998 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:21:19.514 INFO:teuthology.orchestra.run.smithi174.stderr:adjusted tunables profile to default 2019-12-09T11:21:19.527 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-12-09T11:21:19.527 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:19.582 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.OxNMszJidy 2019-12-09T11:21:19.582 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.OxNMszJidy 2019-12-09T11:21:19.646 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.OxNMszJidy 2019-12-09T11:21:19.747 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.OxNMszJidy is 37B 2019-12-09T11:21:19.795 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.OxNMszJidy 2019-12-09T11:21:19.837 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:19.921 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.Cj6rfmCelY 2019-12-09T11:21:19.922 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.Cj6rfmCelY 2019-12-09T11:21:19.987 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.Cj6rfmCelY 2019-12-09T11:21:20.096 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.Cj6rfmCelY is 37B 2019-12-09T11:21:20.143 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.Cj6rfmCelY 2019-12-09T11:21:20.185 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:20.267 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.UJY3Xg7Stx 2019-12-09T11:21:20.268 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.UJY3Xg7Stx 2019-12-09T11:21:20.361 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.UJY3Xg7Stx 2019-12-09T11:21:20.460 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.UJY3Xg7Stx is 37B 2019-12-09T11:21:20.506 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.UJY3Xg7Stx 2019-12-09T11:21:20.548 INFO:teuthology.orchestra.run.smithi106:> mktemp 2019-12-09T11:21:20.638 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.SlWEUl2FL4 2019-12-09T11:21:20.639 INFO:teuthology.orchestra.run.smithi106:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.SlWEUl2FL4 2019-12-09T11:21:20.708 INFO:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.SlWEUl2FL4 2019-12-09T11:21:20.810 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.SlWEUl2FL4 is 37B 2019-12-09T11:21:20.860 INFO:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.SlWEUl2FL4 2019-12-09T11:21:20.904 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:20.967 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.d1uUN5acCy 2019-12-09T11:21:20.967 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.d1uUN5acCy 2019-12-09T11:21:21.073 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.d1uUN5acCy 2019-12-09T11:21:21.200 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.d1uUN5acCy is 37B 2019-12-09T11:21:21.248 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.d1uUN5acCy 2019-12-09T11:21:21.292 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:21.387 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.azAyEnhM8x 2019-12-09T11:21:21.388 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.azAyEnhM8x 2019-12-09T11:21:21.456 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.azAyEnhM8x 2019-12-09T11:21:21.561 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.azAyEnhM8x is 37B 2019-12-09T11:21:21.612 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.azAyEnhM8x 2019-12-09T11:21:21.655 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:21.742 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.lrEP9nScgh 2019-12-09T11:21:21.742 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.lrEP9nScgh 2019-12-09T11:21:21.807 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.lrEP9nScgh 2019-12-09T11:21:21.909 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.lrEP9nScgh is 37B 2019-12-09T11:21:21.958 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.lrEP9nScgh 2019-12-09T11:21:22.004 INFO:teuthology.orchestra.run.smithi174:> mktemp 2019-12-09T11:21:22.105 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.YOQAzirRMi 2019-12-09T11:21:22.106 INFO:teuthology.orchestra.run.smithi174:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.YOQAzirRMi 2019-12-09T11:21:22.174 INFO:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.YOQAzirRMi 2019-12-09T11:21:22.282 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.YOQAzirRMi is 37B 2019-12-09T11:21:22.331 INFO:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.YOQAzirRMi 2019-12-09T11:21:22.377 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new cd9b5090-2f7e-4508-a303-3fa659e8e8e6 0 2019-12-09T11:21:23.702 INFO:teuthology.orchestra.run.smithi174.stdout:0 2019-12-09T11:21:23.728 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 2e98a859-2176-4da8-aaf0-487be2af8838 1 2019-12-09T11:21:24.824 INFO:teuthology.orchestra.run.smithi174.stdout:1 2019-12-09T11:21:24.845 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 727d5aa2-7712-47a8-bf51-945917c45648 2 2019-12-09T11:21:25.766 INFO:teuthology.orchestra.run.smithi174.stdout:2 2019-12-09T11:21:25.789 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new eebe425a-8f44-4920-854d-1690c7f17df3 3 2019-12-09T11:21:26.870 INFO:teuthology.orchestra.run.smithi174.stdout:3 2019-12-09T11:21:26.897 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 05c2bece-6540-4a05-96f9-3e42867a32ce 4 2019-12-09T11:21:27.932 INFO:teuthology.orchestra.run.smithi174.stdout:4 2019-12-09T11:21:27.958 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 8a350448-d92e-46ab-8130-33c006881c15 5 2019-12-09T11:21:28.937 INFO:teuthology.orchestra.run.smithi174.stdout:5 2019-12-09T11:21:28.959 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 0df970bc-64ff-4fbc-9b3c-4cf662d00fbd 6 2019-12-09T11:21:30.028 INFO:teuthology.orchestra.run.smithi174.stdout:6 2019-12-09T11:21:30.050 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd new 4df916f2-63b4-4136-acba-0a55504bbe5b 7 2019-12-09T11:21:31.112 INFO:teuthology.orchestra.run.smithi174.stdout:7 2019-12-09T11:21:31.133 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.134 INFO:tasks.ceph.osd.4:Restarting daemon 2019-12-09T11:21:31.134 INFO:teuthology.orchestra.run.smithi106:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-12-09T11:21:31.137 INFO:tasks.ceph.osd.4:Started 2019-12-09T11:21:31.138 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.138 INFO:tasks.ceph.osd.5:Restarting daemon 2019-12-09T11:21:31.138 INFO:teuthology.orchestra.run.smithi106:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-12-09T11:21:31.141 INFO:tasks.ceph.osd.5:Started 2019-12-09T11:21:31.141 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.141 INFO:tasks.ceph.osd.6:Restarting daemon 2019-12-09T11:21:31.142 INFO:teuthology.orchestra.run.smithi106:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-12-09T11:21:31.144 INFO:tasks.ceph.osd.6:Started 2019-12-09T11:21:31.145 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.145 INFO:tasks.ceph.osd.7:Restarting daemon 2019-12-09T11:21:31.145 INFO:teuthology.orchestra.run.smithi106:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-12-09T11:21:31.148 INFO:tasks.ceph.osd.7:Started 2019-12-09T11:21:31.148 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.149 INFO:tasks.ceph.osd.0:Restarting daemon 2019-12-09T11:21:31.149 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-12-09T11:21:31.159 INFO:tasks.ceph.osd.0:Started 2019-12-09T11:21:31.159 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.159 INFO:tasks.ceph.osd.1:Restarting daemon 2019-12-09T11:21:31.159 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-12-09T11:21:31.166 INFO:tasks.ceph.osd.1:Started 2019-12-09T11:21:31.166 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.166 INFO:tasks.ceph.osd.2:Restarting daemon 2019-12-09T11:21:31.167 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-12-09T11:21:31.174 INFO:tasks.ceph.osd.2:Started 2019-12-09T11:21:31.174 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-12-09T11:21:31.175 INFO:tasks.ceph.osd.3:Restarting daemon 2019-12-09T11:21:31.175 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-12-09T11:21:31.178 INFO:tasks.ceph.osd.3:Started 2019-12-09T11:21:31.178 INFO:tasks.ceph:Waiting for OSDs to come up 2019-12-09T11:21:31.985 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:32.881 INFO:teuthology.misc.health.smithi174.stdout: 2019-12-09T11:21:32.882 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:30.976472+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:32.900 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T11:21:34.936 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:03.662 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:34.963 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:03.678 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:34.984 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:03.700 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:34.993 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:03.718 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.000 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:03.728 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.019 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:03.735 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.028 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:03.723 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.043 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:03.756 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.064 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:03.785 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.078 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:03.787 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.085 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:03.780 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.136 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:03.844 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.576 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T11:21:35.562+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.636 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T11:21:35.622+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.671 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T11:21:35.652+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.680 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T11:21:35.667+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.714 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T11:21:35.700+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T11:21:35.701+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:35.778 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:04.503 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.894 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:04.622 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.904 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:04.612 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.909 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:04.604 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.949 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:04.665 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:35.984 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:04.699 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.001 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:04.710 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.013 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:04.721 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.026 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:04.750 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.034 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:04.741 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.091 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:04.798 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.105 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:04.829 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.162 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:04.887 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.175 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:04.900 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.296 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:05.024 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.308 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:05.036 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.332 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:05.027 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.355 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:05.050 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.361 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:05.076 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.379 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:05.094 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.388 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:05.104 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.400 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:05.116 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.650 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T11:21:36.636+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:36.708 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T11:21:36.694+0000 9b13800 -1 Falling back to public interface 2019-12-09T11:21:36.928 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:05.635 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:36.928 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:05.652 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:37.315 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:06.022 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:37.327 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:06.051 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:37.331 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:06.039 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:37.341 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:06.065 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:38.915 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:07.623 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:38.926 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:07.634 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.047 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:07.776 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.052 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:07.767 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.058 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:07.787 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.061 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:07.777 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.083 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:07.808 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.087 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:07.812 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.191 INFO:tasks.ceph.osd.2.smithi174.stderr:==00:00:00:07.899 10739== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.312 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:08.007 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.322 INFO:tasks.ceph.osd.5.smithi106.stderr:==00:00:00:08.049 9715== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.329 INFO:tasks.ceph.osd.7.smithi106.stderr:==00:00:00:08.043 9718== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.329 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:08.020 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.349 INFO:tasks.ceph.osd.6.smithi106.stderr:==00:00:00:08.074 9716== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.577 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:08.292 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.588 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:08.304 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.591 INFO:tasks.ceph.osd.1.smithi174.stderr:==00:00:00:08.286 10741== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:39.714 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:39.853 INFO:tasks.ceph.osd.4.smithi106.stderr:==00:00:00:08.568 9717== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.586 INFO:teuthology.misc.health.smithi174.stdout: 2019-12-09T11:21:40.587 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":10,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:30.976472+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:40.605 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-12-09T11:21:40.653 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:09.377 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.667 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:09.392 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.669 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:09.376 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.678 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:09.386 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.722 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T11:21:40.722+0000 9b13800 -1 osd.5 0 log_to_monitors {default=true} 2019-12-09T11:21:40.754 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T11:21:40.749+0000 9b13800 -1 osd.2 0 log_to_monitors {default=true} 2019-12-09T11:21:40.895 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T11:21:40.887+0000 9b13800 -1 osd.7 0 log_to_monitors {default=true} 2019-12-09T11:21:40.930 INFO:tasks.ceph.osd.0.smithi174.stderr:==00:00:00:09.655 10736== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.946 INFO:tasks.ceph.osd.3.smithi174.stderr:==00:00:00:09.653 10740== Warning: unimplemented fcntl command: 1038 2019-12-09T11:21:40.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T11:21:40.973+0000 9b13800 -1 osd.6 0 log_to_monitors {default=true} 2019-12-09T11:21:41.109 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T11:21:41.107+0000 9b13800 -1 osd.1 0 log_to_monitors {default=true} 2019-12-09T11:21:41.324 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T11:21:41.320+0000 9b13800 -1 osd.4 0 log_to_monitors {default=true} 2019-12-09T11:21:42.331 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T11:21:42.328+0000 9b13800 -1 osd.0 0 log_to_monitors {default=true} 2019-12-09T11:21:42.340 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T11:21:42.337+0000 9b13800 -1 osd.3 0 log_to_monitors {default=true} 2019-12-09T11:21:43.608 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T11:21:43.605+0000 2476e700 -1 osd.2 0 waiting for initial osdmap 2019-12-09T11:21:43.619 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T11:21:43.607+0000 27772700 -1 osd.7 0 waiting for initial osdmap 2019-12-09T11:21:43.623 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T11:21:43.618+0000 27772700 -1 osd.5 0 waiting for initial osdmap 2019-12-09T11:21:43.642 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T11:21:43.634+0000 2476e700 -1 osd.1 0 waiting for initial osdmap 2019-12-09T11:21:43.649 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T11:21:43.649+0000 27772700 -1 osd.6 0 waiting for initial osdmap 2019-12-09T11:21:43.657 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T11:21:43.655+0000 2476e700 -1 osd.4 0 waiting for initial osdmap 2019-12-09T11:21:44.858 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T11:21:44.852+0000 2476e700 -1 osd.3 0 waiting for initial osdmap 2019-12-09T11:21:44.936 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T11:21:44.932+0000 2476e700 -1 osd.0 0 waiting for initial osdmap 2019-12-09T11:21:47.413 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:48.276 INFO:teuthology.misc.health.smithi174.stdout: 2019-12-09T11:21:48.277 INFO:teuthology.misc.health.smithi174.stdout:{"epoch":16,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:47.577812+0000","last_up_change":"2019-12-09T11:21:46.537032+0000","last_in_change":"2019-12-09T11:21:46.537032+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":10736},{"type":"v1","addr":"172.21.15.174:6817","nonce":10736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":10736},{"type":"v1","addr":"172.21.15.174:6819","nonce":10736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":10736},{"type":"v1","addr":"172.21.15.174:6824","nonce":10736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":10736},{"type":"v1","addr":"172.21.15.174:6821","nonce":10736}]},"public_addr":"172.21.15.174:6817/10736","cluster_addr":"172.21.15.174:6819/10736","heartbeat_back_addr":"172.21.15.174:6824/10736","heartbeat_front_addr":"172.21.15.174:6821/10736","state":["exists","up"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":10741},{"type":"v1","addr":"172.21.15.174:6803","nonce":10741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":10741},{"type":"v1","addr":"172.21.15.174:6807","nonce":10741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":10741},{"type":"v1","addr":"172.21.15.174:6815","nonce":10741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":10741},{"type":"v1","addr":"172.21.15.174:6811","nonce":10741}]},"public_addr":"172.21.15.174:6803/10741","cluster_addr":"172.21.15.174:6807/10741","heartbeat_back_addr":"172.21.15.174:6815/10741","heartbeat_front_addr":"172.21.15.174:6811/10741","state":["exists","up"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":10739},{"type":"v1","addr":"172.21.15.174:6801","nonce":10739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":10739},{"type":"v1","addr":"172.21.15.174:6805","nonce":10739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":10739},{"type":"v1","addr":"172.21.15.174:6813","nonce":10739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":10739},{"type":"v1","addr":"172.21.15.174:6809","nonce":10739}]},"public_addr":"172.21.15.174:6801/10739","cluster_addr":"172.21.15.174:6805/10739","heartbeat_back_addr":"172.21.15.174:6813/10739","heartbeat_front_addr":"172.21.15.174:6809/10739","state":["exists","up"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6823","nonce":10740},{"type":"v1","addr":"172.21.15.174:6825","nonce":10740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":10740},{"type":"v1","addr":"172.21.15.174:6827","nonce":10740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":10740},{"type":"v1","addr":"172.21.15.174:6831","nonce":10740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":10740},{"type":"v1","addr":"172.21.15.174:6829","nonce":10740}]},"public_addr":"172.21.15.174:6825/10740","cluster_addr":"172.21.15.174:6827/10740","heartbeat_back_addr":"172.21.15.174:6831/10740","heartbeat_front_addr":"172.21.15.174:6829/10740","state":["exists","up"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":9717},{"type":"v1","addr":"172.21.15.106:6811","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":9717},{"type":"v1","addr":"172.21.15.106:6813","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":9717},{"type":"v1","addr":"172.21.15.106:6817","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":9717},{"type":"v1","addr":"172.21.15.106:6815","nonce":9717}]},"public_addr":"172.21.15.106:6811/9717","cluster_addr":"172.21.15.106:6813/9717","heartbeat_back_addr":"172.21.15.106:6817/9717","heartbeat_front_addr":"172.21.15.106:6815/9717","state":["exists","up"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":9715},{"type":"v1","addr":"172.21.15.106:6821","nonce":9715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":9715},{"type":"v1","addr":"172.21.15.106:6826","nonce":9715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":9715},{"type":"v1","addr":"172.21.15.106:6833","nonce":9715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":9715},{"type":"v1","addr":"172.21.15.106:6830","nonce":9715}]},"public_addr":"172.21.15.106:6821/9715","cluster_addr":"172.21.15.106:6826/9715","heartbeat_back_addr":"172.21.15.106:6833/9715","heartbeat_front_addr":"172.21.15.106:6830/9715","state":["exists","up"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":9716},{"type":"v1","addr":"172.21.15.106:6803","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":9716},{"type":"v1","addr":"172.21.15.106:6805","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":9716},{"type":"v1","addr":"172.21.15.106:6809","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":9716},{"type":"v1","addr":"172.21.15.106:6807","nonce":9716}]},"public_addr":"172.21.15.106:6803/9716","cluster_addr":"172.21.15.106:6805/9716","heartbeat_back_addr":"172.21.15.106:6809/9716","heartbeat_front_addr":"172.21.15.106:6807/9716","state":["exists","up"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":9718},{"type":"v1","addr":"172.21.15.106:6819","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":9718},{"type":"v1","addr":"172.21.15.106:6823","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6829","nonce":9718},{"type":"v1","addr":"172.21.15.106:6831","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6825","nonce":9718},{"type":"v1","addr":"172.21.15.106:6827","nonce":9718}]},"public_addr":"172.21.15.106:6819/9718","cluster_addr":"172.21.15.106:6823/9718","heartbeat_back_addr":"172.21.15.106:6831/9718","heartbeat_front_addr":"172.21.15.106:6827/9718","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.436791+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.202920+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.875339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.496452+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.477677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.816448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.057763+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.019658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:48.289 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-12-09T11:21:48.289 INFO:tasks.ceph:Creating RBD pool 2019-12-09T11:21:48.290 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd pool create rbd 8 2019-12-09T11:21:50.578 INFO:teuthology.orchestra.run.smithi174.stderr:pool 'rbd' created 2019-12-09T11:21:50.603 INFO:teuthology.orchestra.run.smithi174:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-12-09T11:21:51.733 INFO:teuthology.orchestra.run.smithi174.stderr:enabled application 'rbd' on pool 'rbd' 2019-12-09T11:21:51.746 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-12-09T11:21:51.746 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:51.747 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-12-09T11:21:52.469 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:21:52.469 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":18,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:51.424217+0000","last_up_change":"2019-12-09T11:21:46.537032+0000","last_in_change":"2019-12-09T11:21:46.537032+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T11:21:49.188106+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":10736},{"type":"v1","addr":"172.21.15.174:6817","nonce":10736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":10736},{"type":"v1","addr":"172.21.15.174:6819","nonce":10736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":10736},{"type":"v1","addr":"172.21.15.174:6824","nonce":10736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":10736},{"type":"v1","addr":"172.21.15.174:6821","nonce":10736}]},"public_addr":"172.21.15.174:6817/10736","cluster_addr":"172.21.15.174:6819/10736","heartbeat_back_addr":"172.21.15.174:6824/10736","heartbeat_front_addr":"172.21.15.174:6821/10736","state":["exists","up"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":10741},{"type":"v1","addr":"172.21.15.174:6803","nonce":10741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":10741},{"type":"v1","addr":"172.21.15.174:6807","nonce":10741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":10741},{"type":"v1","addr":"172.21.15.174:6815","nonce":10741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":10741},{"type":"v1","addr":"172.21.15.174:6811","nonce":10741}]},"public_addr":"172.21.15.174:6803/10741","cluster_addr":"172.21.15.174:6807/10741","heartbeat_back_addr":"172.21.15.174:6815/10741","heartbeat_front_addr":"172.21.15.174:6811/10741","state":["exists","up"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":10739},{"type":"v1","addr":"172.21.15.174:6801","nonce":10739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":10739},{"type":"v1","addr":"172.21.15.174:6805","nonce":10739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":10739},{"type":"v1","addr":"172.21.15.174:6813","nonce":10739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":10739},{"type":"v1","addr":"172.21.15.174:6809","nonce":10739}]},"public_addr":"172.21.15.174:6801/10739","cluster_addr":"172.21.15.174:6805/10739","heartbeat_back_addr":"172.21.15.174:6813/10739","heartbeat_front_addr":"172.21.15.174:6809/10739","state":["exists","up"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6823","nonce":10740},{"type":"v1","addr":"172.21.15.174:6825","nonce":10740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":10740},{"type":"v1","addr":"172.21.15.174:6827","nonce":10740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":10740},{"type":"v1","addr":"172.21.15.174:6831","nonce":10740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":10740},{"type":"v1","addr":"172.21.15.174:6829","nonce":10740}]},"public_addr":"172.21.15.174:6825/10740","cluster_addr":"172.21.15.174:6827/10740","heartbeat_back_addr":"172.21.15.174:6831/10740","heartbeat_front_addr":"172.21.15.174:6829/10740","state":["exists","up"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":9717},{"type":"v1","addr":"172.21.15.106:6811","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":9717},{"type":"v1","addr":"172.21.15.106:6813","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":9717},{"type":"v1","addr":"172.21.15.106:6817","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":9717},{"type":"v1","addr":"172.21.15.106:6815","nonce":9717}]},"public_addr":"172.21.15.106:6811/9717","cluster_addr":"172.21.15.106:6813/9717","heartbeat_back_addr":"172.21.15.106:6817/9717","heartbeat_front_addr":"172.21.15.106:6815/9717","state":["exists","up"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":9715},{"type":"v1","addr":"172.21.15.106:6821","nonce":9715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":9715},{"type":"v1","addr":"172.21.15.106:6826","nonce":9715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":9715},{"type":"v1","addr":"172.21.15.106:6833","nonce":9715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":9715},{"type":"v1","addr":"172.21.15.106:6830","nonce":9715}]},"public_addr":"172.21.15.106:6821/9715","cluster_addr":"172.21.15.106:6826/9715","heartbeat_back_addr":"172.21.15.106:6833/9715","heartbeat_front_addr":"172.21.15.106:6830/9715","state":["exists","up"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":9716},{"type":"v1","addr":"172.21.15.106:6803","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":9716},{"type":"v1","addr":"172.21.15.106:6805","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":9716},{"type":"v1","addr":"172.21.15.106:6809","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":9716},{"type":"v1","addr":"172.21.15.106:6807","nonce":9716}]},"public_addr":"172.21.15.106:6803/9716","cluster_addr":"172.21.15.106:6805/9716","heartbeat_back_addr":"172.21.15.106:6809/9716","heartbeat_front_addr":"172.21.15.106:6807/9716","state":["exists","up"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":9718},{"type":"v1","addr":"172.21.15.106:6819","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":9718},{"type":"v1","addr":"172.21.15.106:6823","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6829","nonce":9718},{"type":"v1","addr":"172.21.15.106:6831","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6825","nonce":9718},{"type":"v1","addr":"172.21.15.106:6827","nonce":9718}]},"public_addr":"172.21.15.106:6819/9718","cluster_addr":"172.21.15.106:6823/9718","heartbeat_back_addr":"172.21.15.106:6831/9718","heartbeat_front_addr":"172.21.15.106:6827/9718","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.436791+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.202920+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.875339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.496452+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.477677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.816448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.057763+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.019658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:52.481 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-12-09T11:21:49.188106+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'18', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-12-09T11:21:52.482 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-12-09T11:21:53.161 INFO:teuthology.orchestra.run.smithi174.stdout:pg_num: 8 2019-12-09T11:21:53.174 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-12-09T11:21:53.174 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-12-09T11:21:53.174 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-12-09T11:21:54.010 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:21:54.024 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":8833},{"type":"v1","addr":"172.21.15.106:6801","nonce":8833}]},"active_addr":"172.21.15.106:6801/8833","active_change":"2019-12-09T11:21:20.460295+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4106,"name":"y","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"1","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-12-09T11:21:54.030 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-12-09T11:21:54.030 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-12-09T11:21:54.030 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:54.730 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:21:54.731 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":20,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:54.033466+0000","last_up_change":"2019-12-09T11:21:46.537032+0000","last_in_change":"2019-12-09T11:21:46.537032+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T11:21:49.188106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":10736},{"type":"v1","addr":"172.21.15.174:6817","nonce":10736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":10736},{"type":"v1","addr":"172.21.15.174:6819","nonce":10736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":10736},{"type":"v1","addr":"172.21.15.174:6824","nonce":10736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":10736},{"type":"v1","addr":"172.21.15.174:6821","nonce":10736}]},"public_addr":"172.21.15.174:6817/10736","cluster_addr":"172.21.15.174:6819/10736","heartbeat_back_addr":"172.21.15.174:6824/10736","heartbeat_front_addr":"172.21.15.174:6821/10736","state":["exists","up"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":10741},{"type":"v1","addr":"172.21.15.174:6803","nonce":10741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":10741},{"type":"v1","addr":"172.21.15.174:6807","nonce":10741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":10741},{"type":"v1","addr":"172.21.15.174:6815","nonce":10741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":10741},{"type":"v1","addr":"172.21.15.174:6811","nonce":10741}]},"public_addr":"172.21.15.174:6803/10741","cluster_addr":"172.21.15.174:6807/10741","heartbeat_back_addr":"172.21.15.174:6815/10741","heartbeat_front_addr":"172.21.15.174:6811/10741","state":["exists","up"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":10739},{"type":"v1","addr":"172.21.15.174:6801","nonce":10739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":10739},{"type":"v1","addr":"172.21.15.174:6805","nonce":10739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":10739},{"type":"v1","addr":"172.21.15.174:6813","nonce":10739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":10739},{"type":"v1","addr":"172.21.15.174:6809","nonce":10739}]},"public_addr":"172.21.15.174:6801/10739","cluster_addr":"172.21.15.174:6805/10739","heartbeat_back_addr":"172.21.15.174:6813/10739","heartbeat_front_addr":"172.21.15.174:6809/10739","state":["exists","up"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6823","nonce":10740},{"type":"v1","addr":"172.21.15.174:6825","nonce":10740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":10740},{"type":"v1","addr":"172.21.15.174:6827","nonce":10740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":10740},{"type":"v1","addr":"172.21.15.174:6831","nonce":10740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":10740},{"type":"v1","addr":"172.21.15.174:6829","nonce":10740}]},"public_addr":"172.21.15.174:6825/10740","cluster_addr":"172.21.15.174:6827/10740","heartbeat_back_addr":"172.21.15.174:6831/10740","heartbeat_front_addr":"172.21.15.174:6829/10740","state":["exists","up"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":9717},{"type":"v1","addr":"172.21.15.106:6811","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":9717},{"type":"v1","addr":"172.21.15.106:6813","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":9717},{"type":"v1","addr":"172.21.15.106:6817","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":9717},{"type":"v1","addr":"172.21.15.106:6815","nonce":9717}]},"public_addr":"172.21.15.106:6811/9717","cluster_addr":"172.21.15.106:6813/9717","heartbeat_back_addr":"172.21.15.106:6817/9717","heartbeat_front_addr":"172.21.15.106:6815/9717","state":["exists","up"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":9715},{"type":"v1","addr":"172.21.15.106:6821","nonce":9715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":9715},{"type":"v1","addr":"172.21.15.106:6826","nonce":9715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":9715},{"type":"v1","addr":"172.21.15.106:6833","nonce":9715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":9715},{"type":"v1","addr":"172.21.15.106:6830","nonce":9715}]},"public_addr":"172.21.15.106:6821/9715","cluster_addr":"172.21.15.106:6826/9715","heartbeat_back_addr":"172.21.15.106:6833/9715","heartbeat_front_addr":"172.21.15.106:6830/9715","state":["exists","up"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":9716},{"type":"v1","addr":"172.21.15.106:6803","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":9716},{"type":"v1","addr":"172.21.15.106:6805","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":9716},{"type":"v1","addr":"172.21.15.106:6809","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":9716},{"type":"v1","addr":"172.21.15.106:6807","nonce":9716}]},"public_addr":"172.21.15.106:6803/9716","cluster_addr":"172.21.15.106:6805/9716","heartbeat_back_addr":"172.21.15.106:6809/9716","heartbeat_front_addr":"172.21.15.106:6807/9716","state":["exists","up"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":9718},{"type":"v1","addr":"172.21.15.106:6819","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":9718},{"type":"v1","addr":"172.21.15.106:6823","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6829","nonce":9718},{"type":"v1","addr":"172.21.15.106:6831","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6825","nonce":9718},{"type":"v1","addr":"172.21.15.106:6827","nonce":9718}]},"public_addr":"172.21.15.106:6819/9718","cluster_addr":"172.21.15.106:6823/9718","heartbeat_back_addr":"172.21.15.106:6831/9718","heartbeat_front_addr":"172.21.15.106:6827/9718","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.436791+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.202920+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.875339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.496452+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.477677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.816448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.057763+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.019658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:54.748 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-12-09T11:21:54.748 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-12-09T11:21:55.619 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:21:55.620 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":20,"fsid":"f7df3c5f-47d5-4d9c-8d28-009d70e50517","created":"2019-12-09T11:21:16.569057+0000","modified":"2019-12-09T11:21:54.033466+0000","last_up_change":"2019-12-09T11:21:46.537032+0000","last_in_change":"2019-12-09T11:21:46.537032+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-12-09T11:21:49.188106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"cd9b5090-2f7e-4508-a303-3fa659e8e8e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":10736},{"type":"v1","addr":"172.21.15.174:6817","nonce":10736}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":10736},{"type":"v1","addr":"172.21.15.174:6819","nonce":10736}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":10736},{"type":"v1","addr":"172.21.15.174:6824","nonce":10736}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":10736},{"type":"v1","addr":"172.21.15.174:6821","nonce":10736}]},"public_addr":"172.21.15.174:6817/10736","cluster_addr":"172.21.15.174:6819/10736","heartbeat_back_addr":"172.21.15.174:6824/10736","heartbeat_front_addr":"172.21.15.174:6821/10736","state":["exists","up"]},{"osd":1,"uuid":"2e98a859-2176-4da8-aaf0-487be2af8838","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":10741},{"type":"v1","addr":"172.21.15.174:6803","nonce":10741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":10741},{"type":"v1","addr":"172.21.15.174:6807","nonce":10741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":10741},{"type":"v1","addr":"172.21.15.174:6815","nonce":10741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":10741},{"type":"v1","addr":"172.21.15.174:6811","nonce":10741}]},"public_addr":"172.21.15.174:6803/10741","cluster_addr":"172.21.15.174:6807/10741","heartbeat_back_addr":"172.21.15.174:6815/10741","heartbeat_front_addr":"172.21.15.174:6811/10741","state":["exists","up"]},{"osd":2,"uuid":"727d5aa2-7712-47a8-bf51-945917c45648","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":10739},{"type":"v1","addr":"172.21.15.174:6801","nonce":10739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":10739},{"type":"v1","addr":"172.21.15.174:6805","nonce":10739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":10739},{"type":"v1","addr":"172.21.15.174:6813","nonce":10739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":10739},{"type":"v1","addr":"172.21.15.174:6809","nonce":10739}]},"public_addr":"172.21.15.174:6801/10739","cluster_addr":"172.21.15.174:6805/10739","heartbeat_back_addr":"172.21.15.174:6813/10739","heartbeat_front_addr":"172.21.15.174:6809/10739","state":["exists","up"]},{"osd":3,"uuid":"eebe425a-8f44-4920-854d-1690c7f17df3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6823","nonce":10740},{"type":"v1","addr":"172.21.15.174:6825","nonce":10740}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":10740},{"type":"v1","addr":"172.21.15.174:6827","nonce":10740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":10740},{"type":"v1","addr":"172.21.15.174:6831","nonce":10740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":10740},{"type":"v1","addr":"172.21.15.174:6829","nonce":10740}]},"public_addr":"172.21.15.174:6825/10740","cluster_addr":"172.21.15.174:6827/10740","heartbeat_back_addr":"172.21.15.174:6831/10740","heartbeat_front_addr":"172.21.15.174:6829/10740","state":["exists","up"]},{"osd":4,"uuid":"05c2bece-6540-4a05-96f9-3e42867a32ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":9717},{"type":"v1","addr":"172.21.15.106:6811","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":9717},{"type":"v1","addr":"172.21.15.106:6813","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":9717},{"type":"v1","addr":"172.21.15.106:6817","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":9717},{"type":"v1","addr":"172.21.15.106:6815","nonce":9717}]},"public_addr":"172.21.15.106:6811/9717","cluster_addr":"172.21.15.106:6813/9717","heartbeat_back_addr":"172.21.15.106:6817/9717","heartbeat_front_addr":"172.21.15.106:6815/9717","state":["exists","up"]},{"osd":5,"uuid":"8a350448-d92e-46ab-8130-33c006881c15","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":9715},{"type":"v1","addr":"172.21.15.106:6821","nonce":9715}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":9715},{"type":"v1","addr":"172.21.15.106:6826","nonce":9715}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":9715},{"type":"v1","addr":"172.21.15.106:6833","nonce":9715}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":9715},{"type":"v1","addr":"172.21.15.106:6830","nonce":9715}]},"public_addr":"172.21.15.106:6821/9715","cluster_addr":"172.21.15.106:6826/9715","heartbeat_back_addr":"172.21.15.106:6833/9715","heartbeat_front_addr":"172.21.15.106:6830/9715","state":["exists","up"]},{"osd":6,"uuid":"0df970bc-64ff-4fbc-9b3c-4cf662d00fbd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":9716},{"type":"v1","addr":"172.21.15.106:6803","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":9716},{"type":"v1","addr":"172.21.15.106:6805","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":9716},{"type":"v1","addr":"172.21.15.106:6809","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":9716},{"type":"v1","addr":"172.21.15.106:6807","nonce":9716}]},"public_addr":"172.21.15.106:6803/9716","cluster_addr":"172.21.15.106:6805/9716","heartbeat_back_addr":"172.21.15.106:6809/9716","heartbeat_front_addr":"172.21.15.106:6807/9716","state":["exists","up"]},{"osd":7,"uuid":"4df916f2-63b4-4136-acba-0a55504bbe5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":9718},{"type":"v1","addr":"172.21.15.106:6819","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":9718},{"type":"v1","addr":"172.21.15.106:6823","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6829","nonce":9718},{"type":"v1","addr":"172.21.15.106:6831","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6825","nonce":9718},{"type":"v1","addr":"172.21.15.106:6827","nonce":9718}]},"public_addr":"172.21.15.106:6819/9718","cluster_addr":"172.21.15.106:6823/9718","heartbeat_back_addr":"172.21.15.106:6831/9718","heartbeat_front_addr":"172.21.15.106:6827/9718","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.436791+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.202920+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.875339+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:43.496452+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.477677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:41.816448+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.057763+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-12-09T11:21:42.019658+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-12-09T11:21:55.634 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-12-09T11:21:56.148 INFO:teuthology.orchestra.run.smithi174.stdout:64424509445 2019-12-09T11:21:56.148 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-12-09T11:21:56.643 INFO:teuthology.orchestra.run.smithi174.stdout:60129542149 2019-12-09T11:21:56.644 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-12-09T11:21:57.131 INFO:teuthology.orchestra.run.smithi174.stdout:55834574854 2019-12-09T11:21:57.131 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-12-09T11:21:57.674 INFO:teuthology.orchestra.run.smithi174.stdout:64424509446 2019-12-09T11:21:57.674 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-12-09T11:21:58.251 INFO:teuthology.orchestra.run.smithi174.stdout:60129542151 2019-12-09T11:21:58.252 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-12-09T11:21:58.851 INFO:teuthology.orchestra.run.smithi174.stdout:55834574854 2019-12-09T11:21:58.851 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-12-09T11:21:59.418 INFO:teuthology.orchestra.run.smithi174.stdout:55834574854 2019-12-09T11:21:59.418 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-12-09T11:22:00.034 INFO:teuthology.orchestra.run.smithi174.stdout:55834574852 2019-12-09T11:22:00.034 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-12-09T11:22:00.782 INFO:teuthology.orchestra.run.smithi174.stdout:64424509446 2019-12-09T11:22:00.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.0 2019-12-09T11:22:00.804 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-12-09T11:22:01.549 INFO:teuthology.orchestra.run.smithi174.stdout:60129542150 2019-12-09T11:22:01.564 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2019-12-09T11:22:01.565 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-12-09T11:22:02.515 INFO:teuthology.orchestra.run.smithi174.stdout:55834574855 2019-12-09T11:22:02.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2019-12-09T11:22:02.530 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-12-09T11:22:03.366 INFO:teuthology.orchestra.run.smithi174.stdout:64424509447 2019-12-09T11:22:03.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.3 2019-12-09T11:22:03.381 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-12-09T11:22:04.156 INFO:teuthology.orchestra.run.smithi174.stdout:60129542152 2019-12-09T11:22:04.174 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542152 for osd.4 2019-12-09T11:22:04.175 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-12-09T11:22:04.835 INFO:teuthology.orchestra.run.smithi174.stdout:55834574855 2019-12-09T11:22:04.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2019-12-09T11:22:04.848 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-12-09T11:22:05.615 INFO:teuthology.orchestra.run.smithi174.stdout:55834574855 2019-12-09T11:22:05.629 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.6 2019-12-09T11:22:05.629 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-12-09T11:22:06.321 INFO:teuthology.orchestra.run.smithi174.stdout:55834574853 2019-12-09T11:22:06.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574853 for osd.7 2019-12-09T11:22:06.341 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-12-09T11:22:06.341 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T11:22:07.108 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2019-12-09T11:22:07.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:22:07.122 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-12-09T11:22:06.793802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390208,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584512,"statfs":{"total":773094113280,"available":764502540288,"internally_reserved":8589934592,"allocated":1572864,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.435903+0000","last_change":"2019-12-09T11:21:53.413682+0000","last_active":"2019-12-09T11:21:53.435903+0000","last_peered":"2019-12-09T11:21:53.435903+0000","last_clean":"2019-12-09T11:21:53.435903+0000","last_became_active":"2019-12-09T11:21:53.370947+0000","last_became_peered":"2019-12-09T11:21:53.370947+0000","last_unstale":"2019-12-09T11:21:53.435903+0000","last_undegraded":"2019-12-09T11:21:53.435903+0000","last_fullsized":"2019-12-09T11:21:53.435903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.436516+0000","last_change":"2019-12-09T11:21:53.416112+0000","last_active":"2019-12-09T11:21:53.436516+0000","last_peered":"2019-12-09T11:21:53.436516+0000","last_clean":"2019-12-09T11:21:53.436516+0000","last_became_active":"2019-12-09T11:21:53.368758+0000","last_became_peered":"2019-12-09T11:21:53.368758+0000","last_unstale":"2019-12-09T11:21:53.436516+0000","last_undegraded":"2019-12-09T11:21:53.436516+0000","last_fullsized":"2019-12-09T11:21:53.436516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.224144+0000","last_change":"2019-12-09T11:21:52.666078+0000","last_active":"2019-12-09T11:21:53.224144+0000","last_peered":"2019-12-09T11:21:53.224144+0000","last_clean":"2019-12-09T11:21:53.224144+0000","last_became_active":"2019-12-09T11:21:52.631972+0000","last_became_peered":"2019-12-09T11:21:52.631972+0000","last_unstale":"2019-12-09T11:21:53.224144+0000","last_undegraded":"2019-12-09T11:21:53.224144+0000","last_fullsized":"2019-12-09T11:21:53.224144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.135200+0000","last_change":"2019-12-09T11:21:52.844615+0000","last_active":"2019-12-09T11:21:53.135200+0000","last_peered":"2019-12-09T11:21:53.135200+0000","last_clean":"2019-12-09T11:21:53.135200+0000","last_became_active":"2019-12-09T11:21:52.811417+0000","last_became_peered":"2019-12-09T11:21:52.811417+0000","last_unstale":"2019-12-09T11:21:53.135200+0000","last_undegraded":"2019-12-09T11:21:53.135200+0000","last_fullsized":"2019-12-09T11:21:53.135200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.650070+0000","last_change":"2019-12-09T11:21:53.630490+0000","last_active":"2019-12-09T11:21:53.650070+0000","last_peered":"2019-12-09T11:21:53.650070+0000","last_clean":"2019-12-09T11:21:53.650070+0000","last_became_active":"2019-12-09T11:21:53.619386+0000","last_became_peered":"2019-12-09T11:21:53.619386+0000","last_unstale":"2019-12-09T11:21:53.650070+0000","last_undegraded":"2019-12-09T11:21:53.650070+0000","last_fullsized":"2019-12-09T11:21:53.650070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.462979+0000","last_change":"2019-12-09T11:21:53.457166+0000","last_active":"2019-12-09T11:21:53.462979+0000","last_peered":"2019-12-09T11:21:53.462979+0000","last_clean":"2019-12-09T11:21:53.462979+0000","last_became_active":"2019-12-09T11:21:53.442311+0000","last_became_peered":"2019-12-09T11:21:53.442311+0000","last_unstale":"2019-12-09T11:21:53.462979+0000","last_undegraded":"2019-12-09T11:21:53.462979+0000","last_fullsized":"2019-12-09T11:21:53.462979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.403339+0000","last_change":"2019-12-09T11:21:53.380950+0000","last_active":"2019-12-09T11:21:53.403339+0000","last_peered":"2019-12-09T11:21:53.403339+0000","last_clean":"2019-12-09T11:21:53.403339+0000","last_became_active":"2019-12-09T11:21:53.350271+0000","last_became_peered":"2019-12-09T11:21:53.350271+0000","last_unstale":"2019-12-09T11:21:53.403339+0000","last_undegraded":"2019-12-09T11:21:53.403339+0000","last_fullsized":"2019-12-09T11:21:53.403339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.535722+0000","last_change":"2019-12-09T11:21:53.510352+0000","last_active":"2019-12-09T11:21:53.535722+0000","last_peered":"2019-12-09T11:21:53.535722+0000","last_clean":"2019-12-09T11:21:53.535722+0000","last_became_active":"2019-12-09T11:21:53.470036+0000","last_became_peered":"2019-12-09T11:21:53.470036+0000","last_unstale":"2019-12-09T11:21:53.535722+0000","last_undegraded":"2019-12-09T11:21:53.535722+0000","last_fullsized":"2019-12-09T11:21:53.535722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.811}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.398}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.231}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.723}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.851}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.716}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.767}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.905}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.219}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.152}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.849}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.302}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.588}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.574}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.336}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.062}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.675}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.262}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.656}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.060}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.032}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.851}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.620}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.055}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.090}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.562}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.851}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.193}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.612}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.885}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.023}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.188}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.404}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.495}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.676}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.710}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.330}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.863}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.446}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.900}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.598}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.910}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.243}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.336}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.811}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.449}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.983}]}]}]}} 2019-12-09T11:22:07.124 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-12-09T11:22:08.055 INFO:teuthology.orchestra.run.smithi174.stderr:dumped all 2019-12-09T11:22:08.055 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:22:08.068 INFO:teuthology.orchestra.run.smithi174.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2019-12-09T11:22:06.793802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8390208,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":8388608,"kb_avail":746584512,"statfs":{"total":773094113280,"available":764502540288,"internally_reserved":8589934592,"allocated":1572864,"data_stored":512824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8589934592},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.435903+0000","last_change":"2019-12-09T11:21:53.413682+0000","last_active":"2019-12-09T11:21:53.435903+0000","last_peered":"2019-12-09T11:21:53.435903+0000","last_clean":"2019-12-09T11:21:53.435903+0000","last_became_active":"2019-12-09T11:21:53.370947+0000","last_became_peered":"2019-12-09T11:21:53.370947+0000","last_unstale":"2019-12-09T11:21:53.435903+0000","last_undegraded":"2019-12-09T11:21:53.435903+0000","last_fullsized":"2019-12-09T11:21:53.435903+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.436516+0000","last_change":"2019-12-09T11:21:53.416112+0000","last_active":"2019-12-09T11:21:53.436516+0000","last_peered":"2019-12-09T11:21:53.436516+0000","last_clean":"2019-12-09T11:21:53.436516+0000","last_became_active":"2019-12-09T11:21:53.368758+0000","last_became_peered":"2019-12-09T11:21:53.368758+0000","last_unstale":"2019-12-09T11:21:53.436516+0000","last_undegraded":"2019-12-09T11:21:53.436516+0000","last_fullsized":"2019-12-09T11:21:53.436516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.224144+0000","last_change":"2019-12-09T11:21:52.666078+0000","last_active":"2019-12-09T11:21:53.224144+0000","last_peered":"2019-12-09T11:21:53.224144+0000","last_clean":"2019-12-09T11:21:53.224144+0000","last_became_active":"2019-12-09T11:21:52.631972+0000","last_became_peered":"2019-12-09T11:21:52.631972+0000","last_unstale":"2019-12-09T11:21:53.224144+0000","last_undegraded":"2019-12-09T11:21:53.224144+0000","last_fullsized":"2019-12-09T11:21:53.224144+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.135200+0000","last_change":"2019-12-09T11:21:52.844615+0000","last_active":"2019-12-09T11:21:53.135200+0000","last_peered":"2019-12-09T11:21:53.135200+0000","last_clean":"2019-12-09T11:21:53.135200+0000","last_became_active":"2019-12-09T11:21:52.811417+0000","last_became_peered":"2019-12-09T11:21:52.811417+0000","last_unstale":"2019-12-09T11:21:53.135200+0000","last_undegraded":"2019-12-09T11:21:53.135200+0000","last_fullsized":"2019-12-09T11:21:53.135200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.650070+0000","last_change":"2019-12-09T11:21:53.630490+0000","last_active":"2019-12-09T11:21:53.650070+0000","last_peered":"2019-12-09T11:21:53.650070+0000","last_clean":"2019-12-09T11:21:53.650070+0000","last_became_active":"2019-12-09T11:21:53.619386+0000","last_became_peered":"2019-12-09T11:21:53.619386+0000","last_unstale":"2019-12-09T11:21:53.650070+0000","last_undegraded":"2019-12-09T11:21:53.650070+0000","last_fullsized":"2019-12-09T11:21:53.650070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.462979+0000","last_change":"2019-12-09T11:21:53.457166+0000","last_active":"2019-12-09T11:21:53.462979+0000","last_peered":"2019-12-09T11:21:53.462979+0000","last_clean":"2019-12-09T11:21:53.462979+0000","last_became_active":"2019-12-09T11:21:53.442311+0000","last_became_peered":"2019-12-09T11:21:53.442311+0000","last_unstale":"2019-12-09T11:21:53.462979+0000","last_undegraded":"2019-12-09T11:21:53.462979+0000","last_fullsized":"2019-12-09T11:21:53.462979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.403339+0000","last_change":"2019-12-09T11:21:53.380950+0000","last_active":"2019-12-09T11:21:53.403339+0000","last_peered":"2019-12-09T11:21:53.403339+0000","last_clean":"2019-12-09T11:21:53.403339+0000","last_became_active":"2019-12-09T11:21:53.350271+0000","last_became_peered":"2019-12-09T11:21:53.350271+0000","last_unstale":"2019-12-09T11:21:53.403339+0000","last_undegraded":"2019-12-09T11:21:53.403339+0000","last_fullsized":"2019-12-09T11:21:53.403339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-12-09T11:21:53.535722+0000","last_change":"2019-12-09T11:21:53.510352+0000","last_active":"2019-12-09T11:21:53.535722+0000","last_peered":"2019-12-09T11:21:53.535722+0000","last_clean":"2019-12-09T11:21:53.535722+0000","last_became_active":"2019-12-09T11:21:53.470036+0000","last_became_peered":"2019-12-09T11:21:53.470036+0000","last_unstale":"2019-12-09T11:21:53.535722+0000","last_undegraded":"2019-12-09T11:21:53.535722+0000","last_fullsized":"2019-12-09T11:21:53.535722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-12-09T11:21:50.105172+0000","last_clean_scrub_stamp":"2019-12-09T11:21:50.105172+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.811}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.398}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.231}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.723}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.097}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.851}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.716}]}]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.767}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":34.905}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.219}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.152}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.849}]}]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.302}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.588}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.574}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.336}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.062}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.675}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.262}]}]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.656}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.060}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.032}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.498}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.436}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.851}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.620}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":7.055}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":9.090}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.438}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.562}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.851}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.193}]}]},{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.612}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.885}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.046}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.023}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.188}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":33.404}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.495}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.676}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.710}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.330}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.863}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.446}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.900}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.598}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1048776,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":1048576,"kb_avail":93323064,"statfs":{"total":96636764160,"available":95562817536,"internally_reserved":1073741824,"allocated":196608,"data_stored":64103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1073741824},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.910}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.243}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.336}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.811}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":42.449}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":36.744}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":30.983}]}]}]}} 2019-12-09T11:22:08.070 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-12-09T11:22:08.070 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-12-09T11:22:08.070 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-12-09T11:22:08.070 INFO:teuthology.orchestra.run.smithi174:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-12-09T11:22:08.831 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:22:08.845 INFO:teuthology.orchestra.run.smithi174.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-12-09T11:22:08.845 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-12-09T11:22:08.845 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-12-09T11:22:08.858 INFO:teuthology.run_tasks:Running task rgw... 2019-12-09T11:22:08.872 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-12-09T11:22:08.872 DEBUG:tasks.rgw:client list is ['client.0'] 2019-12-09T11:22:08.873 INFO:tasks.rgw:Creating data pools 2019-12-09T11:22:08.873 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-12-09T11:22:08.873 INFO:teuthology.orchestra.run.smithi174:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-12-09T11:22:10.263 INFO:teuthology.orchestra.run.smithi174:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-12-09T11:22:13.257 INFO:teuthology.orchestra.run.smithi174.stderr:pool 'default.rgw.buckets.data' created 2019-12-09T11:22:13.275 INFO:teuthology.orchestra.run.smithi174:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-12-09T11:22:14.805 INFO:teuthology.orchestra.run.smithi174.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-12-09T11:22:14.821 INFO:teuthology.orchestra.run.smithi174:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-12-09T11:22:17.004 INFO:teuthology.orchestra.run.smithi174.stderr:pool 'default.rgw.buckets.index' created 2019-12-09T11:22:17.022 INFO:teuthology.orchestra.run.smithi174:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-12-09T11:22:18.127 INFO:teuthology.orchestra.run.smithi174.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-12-09T11:22:18.143 DEBUG:tasks.rgw:Pools created 2019-12-09T11:22:18.143 INFO:tasks.rgw:Configuring compression type = random 2019-12-09T11:22:18.143 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T11:22:18.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T11:22:18.144 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T11:22:18.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:18.340+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:21.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.211+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:21.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.212+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:21.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.239+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:21.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.239+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:21.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.245+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:21.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.245+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:21.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:21.380+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.464 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.466+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.476 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.478+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.476 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.478+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.484+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.484+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.495+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.495+0000 7f37c702aec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:22:23.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.495+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.500 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.502+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.502+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.511+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:23.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.511+0000 7f37c702aec0 10 Creating default zonegroup 2019-12-09T11:22:23.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.511+0000 7f37c702aec0 20 get_system_obj_state: rctx=0x7ffdcf630ee0 obj=:.pools.avail state=0x56332255bbe0 s->prefetch_data=0 2019-12-09T11:22:23.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.515+0000 7f37c702aec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-12-09T11:22:23.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.953+0000 7f37c702aec0 10 WARNING: store->list_zones() returned r=-2 2019-12-09T11:22:23.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.953+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:23.960 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:23.962+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:24.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.928+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:24.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.936+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:24.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.936+0000 7f37c702aec0 10 could not read realm id: (2) No such file or directory 2019-12-09T11:22:24.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.936+0000 7f37c702aec0 10 WARNING: failed to set zone as default, r=-22 2019-12-09T11:22:24.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.936+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:24.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:24.945+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:25.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.494+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:25.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.543+0000 7f37b4a55700 0 -- 172.21.15.174:0/3514985051 >> [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] conn(0x7f37a80069f0 msgr2=0x7f37a80083b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:22:25.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.544+0000 7f37c702aec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T11:22:25.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.544+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:25.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.586+0000 7f37c702aec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T11:22:25.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.586+0000 7f37c702aec0 20 zonegroup default 2019-12-09T11:22:25.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.586+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:25.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.599+0000 7f37c702aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:25.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.599+0000 7f37c702aec0 10 Cannot find current period zone using local zone 2019-12-09T11:22:25.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.599+0000 7f37c702aec0 10 Using default name default 2019-12-09T11:22:25.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.599+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:25.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.657+0000 7f37c702aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:25.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.657+0000 7f37c702aec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:25.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.675+0000 7f37c702aec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T11:22:25.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.675+0000 7f37c702aec0 20 zone default found 2019-12-09T11:22:25.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.675+0000 7f37c702aec0 20 started sync module instance, tier type = 2019-12-09T11:22:25.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:25.675+0000 7f37c702aec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:22:28.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:28.976+0000 7f37b4a55700 0 -- 172.21.15.174:0/3514985051 >> [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] conn(0x7f37a002daa0 msgr2=0x7f37a002fec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=0 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=1 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=2 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=3 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=4 2019-12-09T11:22:29.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=5 2019-12-09T11:22:29.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.482+0000 7f37c702aec0 20 add_watcher() i=6 2019-12-09T11:22:29.521 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.523+0000 7f37c702aec0 20 add_watcher() i=7 2019-12-09T11:22:29.522 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.523+0000 7f37c702aec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:22:29.522 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:29.523+0000 7f3784fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:22:34.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:34.202+0000 7f37c702aec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-12-09T11:22:38.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:38.699+0000 7f37c702aec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-12-09T11:22:43.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.423+0000 7f36eeffd700 20 reqs_thread_entry: start 2019-12-09T11:22:43.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.424+0000 7f36ecff9700 20 reqs_thread_entry: start 2019-12-09T11:22:43.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.424+0000 7f37c702aec0 20 init_complete bucket index max shards: 0 2019-12-09T11:22:43.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.425+0000 7f36daffd700 20 reqs_thread_entry: start 2019-12-09T11:22:43.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.653+0000 7f37c702aec0 20 remove_watcher() i=0 2019-12-09T11:22:43.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.653+0000 7f37c702aec0 2 removed watcher, disabling cache 2019-12-09T11:22:43.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.747+0000 7f37c702aec0 20 remove_watcher() i=1 2019-12-09T11:22:43.858 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.859+0000 7f37c702aec0 20 remove_watcher() i=2 2019-12-09T11:22:43.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:43.943+0000 7f37c702aec0 20 remove_watcher() i=3 2019-12-09T11:22:44.011 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.013+0000 7f37c702aec0 20 remove_watcher() i=4 2019-12-09T11:22:44.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.085+0000 7f37c702aec0 20 remove_watcher() i=5 2019-12-09T11:22:44.182 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.183+0000 7f37c702aec0 20 remove_watcher() i=6 2019-12-09T11:22:44.245 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.246+0000 7f37c702aec0 20 remove_watcher() i=7 2019-12-09T11:22:44.255 INFO:teuthology.orchestra.run.smithi174.stdout:[] 2019-12-09T11:22:44.255 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T11:22:44.255 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T11:22:44.256 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-12-09T11:22:44.256 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-12-09T11:22:44.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.475+0000 7ff0fb7fe700 20 reqs_thread_entry: start 2019-12-09T11:22:44.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.475+0000 7ff1c63a5ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.500+0000 7ff1c63a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:44.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.500+0000 7ff1c63a5ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.511+0000 7ff1c63a5ec0 20 rados_obj.operate() r=0 bl.length=805 2019-12-09T11:22:44.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.512+0000 7ff1c63a5ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.532+0000 7ff1c63a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:44.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.532+0000 7ff1c63a5ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.548+0000 7ff1c63a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:44.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.548+0000 7ff1c63a5ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.562+0000 7ff1c63a5ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T11:22:44.699 INFO:teuthology.orchestra.run.smithi174.stdout:{"id":"7b90ce1f-2e6d-4ddb-b794-a516be24e604","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T11:22:44.699 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'7b90ce1f-2e6d-4ddb-b794-a516be24e604', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T11:22:44.699 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-12-09T11:22:44.700 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-12-09T11:22:44.700 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-12-09T11:22:44.887 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.889+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.927+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:44.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.927+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.935+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:44.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.935+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.946 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.948+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:44.946 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.948+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:44.969 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:44.971+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:45.231 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.233+0000 7f3d3e4d6ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:22:45.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.233+0000 7f3d3e4d6ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:22:45.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.233+0000 7f3d3e4d6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:22:45.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.233+0000 7f3d3e4d6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:22:45.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.233+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.245+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:45.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.245+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.264+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T11:22:45.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.264+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.275 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.277+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:45.275 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.277+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.294+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T11:22:45.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.294+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.306+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:45.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.306+0000 7f3d3e4d6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:22:45.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.306+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.315+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:45.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.315+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.326+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:45.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.326+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.338+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T11:22:45.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.338+0000 7f3d3e4d6ec0 20 zonegroup default 2019-12-09T11:22:45.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.338+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.350+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:45.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.350+0000 7f3d3e4d6ec0 10 Cannot find current period zone using local zone 2019-12-09T11:22:45.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.350+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.360+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:45.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.360+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.377 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.379+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:45.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.379+0000 7f3d3e4d6ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:45.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.399+0000 7f3d3e4d6ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T11:22:45.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.399+0000 7f3d3e4d6ec0 20 zone default found 2019-12-09T11:22:45.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.399+0000 7f3d3e4d6ec0 20 started sync module instance, tier type = 2019-12-09T11:22:45.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.399+0000 7f3d3e4d6ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:22:45.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=0 2019-12-09T11:22:45.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=1 2019-12-09T11:22:45.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=2 2019-12-09T11:22:45.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=3 2019-12-09T11:22:45.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=4 2019-12-09T11:22:45.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=5 2019-12-09T11:22:45.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.919+0000 7f3d3e4d6ec0 20 add_watcher() i=6 2019-12-09T11:22:45.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.968+0000 7f3d3e4d6ec0 20 add_watcher() i=7 2019-12-09T11:22:45.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.968+0000 7f3d3e4d6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:22:45.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.969+0000 7f3cfbfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:22:45.989 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:45.991+0000 7f3d2aeff700 0 -- 172.21.15.174:0/2495697147 >> v1:172.21.15.106:6789/0 conn(0x558e3a4fe460 legacy=0x558e3a500970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:22:46.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:46.520+0000 7f3d2b700700 0 -- 172.21.15.174:0/2495697147 >> [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] conn(0x558e3a510700 msgr2=0x558e3a512bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:22:48.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.564+0000 7f3c6a7fc700 20 reqs_thread_entry: start 2019-12-09T11:22:48.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.564+0000 7f3c63fff700 20 reqs_thread_entry: start 2019-12-09T11:22:48.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.564+0000 7f3d3e4d6ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:22:48.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.565+0000 7f3c627fc700 20 reqs_thread_entry: start 2019-12-09T11:22:48.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.806+0000 7f3d3e4d6ec0 20 remove_watcher() i=0 2019-12-09T11:22:48.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.807+0000 7f3d3e4d6ec0 2 removed watcher, disabling cache 2019-12-09T11:22:48.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.874+0000 7f3d3e4d6ec0 20 remove_watcher() i=1 2019-12-09T11:22:48.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.942+0000 7f3d3e4d6ec0 20 remove_watcher() i=2 2019-12-09T11:22:48.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:48.993+0000 7f3d3e4d6ec0 20 remove_watcher() i=3 2019-12-09T11:22:49.049 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.050+0000 7f3d3e4d6ec0 20 remove_watcher() i=4 2019-12-09T11:22:49.121 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.123+0000 7f3d3e4d6ec0 20 remove_watcher() i=5 2019-12-09T11:22:49.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.173+0000 7f3d3e4d6ec0 20 remove_watcher() i=6 2019-12-09T11:22:49.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.222+0000 7f3d3e4d6ec0 20 remove_watcher() i=7 2019-12-09T11:22:49.231 INFO:teuthology.orchestra.run.smithi174.stdout:[] 2019-12-09T11:22:49.231 DEBUG:tasks.util.rgw: json result: [] 2019-12-09T11:22:49.231 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-12-09T11:22:49.231 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T11:22:49.231 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-12-09T11:22:49.232 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-12-09T11:22:49.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.412+0000 7f1884ff9700 20 reqs_thread_entry: start 2019-12-09T11:22:49.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.412+0000 7f1947626ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.428+0000 7f1947626ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:49.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.428+0000 7f1947626ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.447 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.448+0000 7f1947626ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:49.447 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.449+0000 7f1947626ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.462+0000 7f1947626ec0 20 rados_obj.operate() r=0 bl.length=348 2019-12-09T11:22:49.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.462+0000 7f1947626ec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.474+0000 7f1947626ec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T11:22:49.621 INFO:teuthology.orchestra.run.smithi174.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-12-09T11:22:49.622 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T11:22:49.622 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T11:22:49.622 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-12-09T11:22:49.622 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-12-09T11:22:49.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.930+0000 7fbe4a7fc700 20 reqs_thread_entry: start 2019-12-09T11:22:49.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.930+0000 7fbf1511cec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.953 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.954+0000 7fbf1511cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:49.953 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.954+0000 7fbf1511cec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.968+0000 7fbf1511cec0 20 rados_obj.operate() r=0 bl.length=821 2019-12-09T11:22:49.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.968+0000 7fbf1511cec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.983+0000 7fbf1511cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:49.983 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.983+0000 7fbf1511cec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:49.997 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.998+0000 7fbf1511cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:49.997 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:49.998+0000 7fbf1511cec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.022 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.023+0000 7fbf1511cec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T11:22:50.108 INFO:teuthology.orchestra.run.smithi174.stdout:{"id":"7b90ce1f-2e6d-4ddb-b794-a516be24e604","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T11:22:50.109 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'7b90ce1f-2e6d-4ddb-b794-a516be24e604', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T11:22:50.109 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-12-09T11:22:50.109 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T11:22:50.109 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-12-09T11:22:50.110 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-12-09T11:22:50.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.319+0000 7f784bfff700 20 reqs_thread_entry: start 2019-12-09T11:22:50.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.319+0000 7f791726dec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.341+0000 7f791726dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:50.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.341+0000 7f791726dec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.362+0000 7f791726dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:50.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.362+0000 7f791726dec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.381+0000 7f791726dec0 20 rados_obj.operate() r=0 bl.length=372 2019-12-09T11:22:50.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.381+0000 7f791726dec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.404+0000 7f791726dec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T11:22:50.466 INFO:teuthology.orchestra.run.smithi174.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-12-09T11:22:50.466 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-12-09T11:22:50.466 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T11:22:50.466 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-12-09T11:22:50.467 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-12-09T11:22:50.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.636+0000 7efcfffff700 20 reqs_thread_entry: start 2019-12-09T11:22:50.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.636+0000 7efdcf51fec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.655+0000 7efdcf51fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:50.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.655+0000 7efdcf51fec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.665+0000 7efdcf51fec0 20 rados_obj.operate() r=0 bl.length=888 2019-12-09T11:22:50.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.665+0000 7efdcf51fec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.678+0000 7efdcf51fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:22:50.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.678+0000 7efdcf51fec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.692+0000 7efdcf51fec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:22:50.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.692+0000 7efdcf51fec0 20 rados->read ofs=0 len=0 2019-12-09T11:22:50.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:22:50.712+0000 7efdcf51fec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:22:50.762 INFO:teuthology.orchestra.run.smithi174.stdout:{"id":"7b90ce1f-2e6d-4ddb-b794-a516be24e604","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-12-09T11:22:50.762 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'7b90ce1f-2e6d-4ddb-b794-a516be24e604', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-12-09T11:22:50.762 INFO:tasks.rgw:Starting rgw... 2019-12-09T11:22:50.763 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-12-09T11:22:50.763 INFO:tasks.rgw:Using beast as radosgw frontend 2019-12-09T11:22:50.763 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-12-09T11:22:50.763 INFO:tasks.rgw.client.0:Restarting daemon 2019-12-09T11:22:50.764 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-12-09T11:22:50.766 INFO:tasks.rgw.client.0:Started 2019-12-09T11:22:50.767 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:22:50.767 INFO:teuthology.orchestra.run.smithi174:> curl http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:22:50.863 INFO:teuthology.orchestra.run.smithi174.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T11:22:50.863 INFO:teuthology.orchestra.run.smithi174.stderr: Dload Upload Total Spent Left Speed 2019-12-09T11:22:50.867 INFO:teuthology.orchestra.run.smithi174.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi174.front.sepia.ceph.com:80; Connection refused 2019-12-09T11:22:50.867 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T11:22:51.868 INFO:teuthology.orchestra.run.smithi174:> curl http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:22:51.953 INFO:teuthology.orchestra.run.smithi174.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T11:22:51.953 INFO:teuthology.orchestra.run.smithi174.stderr: Dload Upload Total Spent Left Speed 2019-12-09T11:22:51.956 INFO:teuthology.orchestra.run.smithi174.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi174.front.sepia.ceph.com:80; Connection refused 2019-12-09T11:22:51.956 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T11:22:53.958 INFO:teuthology.orchestra.run.smithi174:> curl http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:22:54.024 INFO:teuthology.orchestra.run.smithi174.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T11:22:54.024 INFO:teuthology.orchestra.run.smithi174.stderr: Dload Upload Total Spent Left Speed 2019-12-09T11:22:54.024 INFO:teuthology.orchestra.run.smithi174.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi174.front.sepia.ceph.com:80; Connection refused 2019-12-09T11:22:54.024 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T11:22:58.025 INFO:teuthology.orchestra.run.smithi174:> curl http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:22:58.108 INFO:teuthology.orchestra.run.smithi174.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T11:22:58.108 INFO:teuthology.orchestra.run.smithi174.stderr: Dload Upload Total Spent Left Speed 2019-12-09T11:22:58.112 INFO:teuthology.orchestra.run.smithi174.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi174.front.sepia.ceph.com:80; Connection refused 2019-12-09T11:22:58.113 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-12-09T11:23:06.114 INFO:teuthology.orchestra.run.smithi174:> curl http://smithi174.front.sepia.ceph.com:80/ 2019-12-09T11:23:06.200 INFO:teuthology.orchestra.run.smithi174.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-12-09T11:23:06.200 INFO:teuthology.orchestra.run.smithi174.stderr: Dload Upload Total Spent Left Speed 2019-12-09T11:23:06.905 INFO:teuthology.orchestra.run.smithi174.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 303 0 --:--:-- --:--:-- --:--:-- 303 2019-12-09T11:23:06.905 INFO:teuthology.orchestra.run.smithi174.stdout:anonymous 2019-12-09T11:23:06.906 INFO:teuthology.run_tasks:Running task workunit... 2019-12-09T11:23:06.945 INFO:tasks.workunit:Pulling workunits from ref 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:23:06.945 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-12-09T11:23:06.945 INFO:teuthology.orchestra.run.smithi174:> stat -- /home/ubuntu/cephtest/mnt.0 2019-12-09T11:23:07.004 INFO:teuthology.orchestra.run.smithi174.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-12-09T11:23:07.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T11:23:07.004 INFO:teuthology.orchestra.run.smithi174:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T11:23:07.049 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-12-09T11:23:07.049 INFO:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-12-09T11:23:07.098 INFO:tasks.workunit:timeout=3h 2019-12-09T11:23:07.099 INFO:tasks.workunit:cleanup=True 2019-12-09T11:23:07.099 INFO:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1351c819e1a9518cde4c8b12d645782009028360 2019-12-09T11:23:07.375 INFO:tasks.workunit.client.0.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-12-09T11:24:12.989 INFO:tasks.workunit.client.0.smithi174.stderr:Note: checking out '1351c819e1a9518cde4c8b12d645782009028360'. 2019-12-09T11:24:12.989 INFO:tasks.workunit.client.0.smithi174.stderr: 2019-12-09T11:24:12.989 INFO:tasks.workunit.client.0.smithi174.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-12-09T11:24:12.989 INFO:tasks.workunit.client.0.smithi174.stderr:changes and commit them, and you can discard any commits you make in this 2019-12-09T11:24:12.989 INFO:tasks.workunit.client.0.smithi174.stderr:state without impacting any branches by performing another checkout. 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr: 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr:If you want to create a new branch to retain commits you create, you may 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr: 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr: git checkout -b new_branch_name 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr: 2019-12-09T11:24:12.990 INFO:tasks.workunit.client.0.smithi174.stderr:HEAD is now at 1351c81... rgw: Fix dynamic resharding not working for empty zonegroup in period 2019-12-09T11:24:12.992 INFO:teuthology.orchestra.run.smithi174:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-12-09T11:24:13.046 INFO:tasks.workunit.client.0.smithi174.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-12-09T11:24:13.048 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T11:24:13.048 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-12-09T11:24:13.566 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-12-09T11:24:13.623 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-12-09T11:24:13.657 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-12-09T11:24:13.659 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T11:24:13.659 INFO:tasks.workunit.client.0.smithi174.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-12-09T11:24:13.700 INFO:tasks.workunit.client.0.smithi174.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-12-09T11:24:13.754 DEBUG:teuthology.orchestra.remote:smithi174:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-12-09T11:24:13.801 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-12-09T11:24:13.801 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-12-09T11:24:13.801 INFO:teuthology.orchestra.run.smithi174:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-12-09T11:24:13.856 INFO:tasks.workunit.client.0.smithi174.stdout:Running main() from gmock_main.cc 2019-12-09T11:24:13.856 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] Running 11 tests from 1 test suite. 2019-12-09T11:24:13.857 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment set-up. 2019-12-09T11:24:13.857 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 11 tests from ClsLock 2019-12-09T11:24:13.857 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-12-09T11:24:15.957 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:15.957 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: exclusive 2019-12-09T11:24:15.957 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:15.957 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:15.958 INFO:tasks.workunit.client.0.smithi174.stdout: client.4504 expiration=2019-12-09T11:26:15.845017+0000 addr=v1:172.21.15.174:0/2142118322 cookie= 2019-12-09T11:24:16.088 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:16.089 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: shared 2019-12-09T11:24:16.089 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:16.089 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:16.089 INFO:tasks.workunit.client.0.smithi174.stdout: client.4470 expiration=0.000000 addr=v1:172.21.15.174:0/192801489 cookie= 2019-12-09T11:24:16.089 INFO:tasks.workunit.client.0.smithi174.stdout: client.4504 expiration=2019-12-09T11:26:16.049849+0000 addr=v1:172.21.15.174:0/2142118322 cookie= 2019-12-09T11:24:16.133 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:16.133 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: shared 2019-12-09T11:24:16.133 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:16.134 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:16.134 INFO:tasks.workunit.client.0.smithi174.stdout: client.4470 expiration=0.000000 addr=v1:172.21.15.174:0/192801489 cookie= 2019-12-09T11:24:17.446 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestMultiLocking (3590 ms) 2019-12-09T11:24:17.446 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestMeta 2019-12-09T11:24:20.487 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:20.487 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: shared 2019-12-09T11:24:20.487 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:20.487 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:20.488 INFO:tasks.workunit.client.0.smithi174.stdout: client.4400 expiration=0.000000 addr=v1:172.21.15.174:0/1492124938 cookie= 2019-12-09T11:24:20.561 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:20.562 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: exclusive 2019-12-09T11:24:20.562 INFO:tasks.workunit.client.0.smithi174.stdout: tag: new_tag 2019-12-09T11:24:20.562 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:20.562 INFO:tasks.workunit.client.0.smithi174.stdout: client.4519 expiration=0.000000 addr=v1:172.21.15.174:0/3613952051 cookie= 2019-12-09T11:24:20.964 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestMeta (3516 ms) 2019-12-09T11:24:20.965 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestCookie 2019-12-09T11:24:23.992 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:23.992 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: exclusive 2019-12-09T11:24:23.992 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:23.993 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:24.065 INFO:tasks.workunit.client.0.smithi174.stdout:lock: mylock 2019-12-09T11:24:24.065 INFO:tasks.workunit.client.0.smithi174.stdout: lock_type: shared 2019-12-09T11:24:24.065 INFO:tasks.workunit.client.0.smithi174.stdout: tag: 2019-12-09T11:24:24.065 INFO:tasks.workunit.client.0.smithi174.stdout: lockers: 2019-12-09T11:24:24.066 INFO:tasks.workunit.client.0.smithi174.stdout: client.4531 expiration=0.000000 addr=v1:172.21.15.174:0/796715958 cookie= 2019-12-09T11:24:24.066 INFO:tasks.workunit.client.0.smithi174.stdout: client.4531 expiration=0.000000 addr=v1:172.21.15.174:0/796715958 cookie=new cookie 2019-12-09T11:24:24.511 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestCookie (3549 ms) 2019-12-09T11:24:24.511 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-12-09T11:24:28.000 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestMultipleLocks (3489 ms) 2019-12-09T11:24:28.001 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestLockDuration 2019-12-09T11:24:36.269 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestLockDuration (8269 ms) 2019-12-09T11:24:36.269 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-12-09T11:24:39.769 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestAssertLocked (3500 ms) 2019-12-09T11:24:39.769 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestSetCookie 2019-12-09T11:24:43.399 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestSetCookie (3630 ms) 2019-12-09T11:24:43.400 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestRenew 2019-12-09T11:25:05.529 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestRenew (22129 ms) 2019-12-09T11:25:05.529 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-12-09T11:25:12.820 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7292 ms) 2019-12-09T11:25:12.820 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-12-09T11:25:20.871 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8051 ms) 2019-12-09T11:25:20.871 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-12-09T11:25:29.856 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8985 ms) 2019-12-09T11:25:29.856 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 11 tests from ClsLock (76000 ms total) 2019-12-09T11:25:29.856 INFO:tasks.workunit.client.0.smithi174.stdout: 2019-12-09T11:25:29.856 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment tear-down 2019-12-09T11:25:29.856 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] 11 tests from 1 test suite ran. (76000 ms total) 2019-12-09T11:25:29.857 INFO:tasks.workunit.client.0.smithi174.stdout:[ PASSED ] 11 tests. 2019-12-09T11:25:29.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T11:25:29.859 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T11:25:30.016 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-12-09T11:25:30.016 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-12-09T11:25:30.017 INFO:teuthology.orchestra.run.smithi174:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-12-09T11:25:30.092 INFO:tasks.workunit.client.0.smithi174.stdout:Running main() from gmock_main.cc 2019-12-09T11:25:30.092 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] Running 4 tests from 1 test suite. 2019-12-09T11:25:30.092 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment set-up. 2019-12-09T11:25:30.092 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 4 tests from cls_log 2019-12-09T11:25:30.092 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-12-09T11:25:33.389 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_log.test_log_add_same_time (3298 ms) 2019-12-09T11:25:33.390 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-12-09T11:25:37.056 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_log.test_log_add_different_time (3667 ms) 2019-12-09T11:25:37.057 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_log.trim_by_time 2019-12-09T11:25:41.666 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_log.trim_by_time (4610 ms) 2019-12-09T11:25:41.666 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_log.trim_by_marker 2019-12-09T11:25:46.129 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_log.trim_by_marker (4463 ms) 2019-12-09T11:25:46.129 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 4 tests from cls_log (16038 ms total) 2019-12-09T11:25:46.130 INFO:tasks.workunit.client.0.smithi174.stdout: 2019-12-09T11:25:46.130 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment tear-down 2019-12-09T11:25:46.130 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] 4 tests from 1 test suite ran. (16038 ms total) 2019-12-09T11:25:46.130 INFO:tasks.workunit.client.0.smithi174.stdout:[ PASSED ] 4 tests. 2019-12-09T11:25:46.132 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T11:25:46.132 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T11:25:46.200 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-12-09T11:25:46.200 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-12-09T11:25:46.201 INFO:teuthology.orchestra.run.smithi174:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-12-09T11:25:46.262 INFO:tasks.workunit.client.0.smithi174.stdout:Running main() from gmock_main.cc 2019-12-09T11:25:46.262 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] Running 5 tests from 1 test suite. 2019-12-09T11:25:46.262 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment set-up. 2019-12-09T11:25:46.262 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 5 tests from cls_rgw 2019-12-09T11:25:46.262 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.test_implicit 2019-12-09T11:25:49.717 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.test_implicit (3456 ms) 2019-12-09T11:25:49.718 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-12-09T11:25:53.143 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3423 ms) 2019-12-09T11:25:53.143 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.test_put_snap 2019-12-09T11:26:02.207 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T11:26:02.148+0000 38794700 -1 osd.5 pg_epoch: 106 pg[25.6( v 105'4 (0'0,105'4] local-lis/les=102/103 n=2 ec=102/102 lis/c=102/102 les/c/f=103/103/0 sis=102) [5,1] r=0 lpr=102 crt=104'3 lcod 104'3 mlcod 104'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-12-09T11:26:03.495 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.test_put_snap (10355 ms) 2019-12-09T11:26:03.496 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.test_explicit 2019-12-09T11:26:07.111 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.test_explicit (3616 ms) 2019-12-09T11:26:07.111 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.set 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.set (3675 ms) 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 5 tests from cls_rgw (24525 ms total) 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout: 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment tear-down 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] 5 tests from 1 test suite ran. (24525 ms total) 2019-12-09T11:26:10.787 INFO:tasks.workunit.client.0.smithi174.stdout:[ PASSED ] 5 tests. 2019-12-09T11:26:10.789 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T11:26:10.789 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T11:26:10.907 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-12-09T11:26:10.907 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-12-09T11:26:10.908 INFO:teuthology.orchestra.run.smithi174:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-12-09T11:26:10.962 INFO:tasks.workunit.client.0.smithi174.stdout:Running main() from gmock_main.cc 2019-12-09T11:26:10.963 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] Running 13 tests from 1 test suite. 2019-12-09T11:26:10.963 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment set-up. 2019-12-09T11:26:10.963 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 13 tests from cls_rgw 2019-12-09T11:26:13.244 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.index_basic 2019-12-09T11:26:14.905 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.index_basic (1661 ms) 2019-12-09T11:26:14.905 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-12-09T11:26:16.471 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (1565 ms) 2019-12-09T11:26:16.471 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.index_remove_object 2019-12-09T11:26:18.635 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.index_remove_object (2165 ms) 2019-12-09T11:26:18.636 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.index_suggest 2019-12-09T11:26:44.600 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.index_suggest (25965 ms) 2019-12-09T11:26:44.600 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.index_list 2019-12-09T11:26:46.241 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.index_list (1641 ms) 2019-12-09T11:26:46.241 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.bi_list 2019-12-09T11:26:49.229 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.bi_list (2989 ms) 2019-12-09T11:26:49.230 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.gc_set 2019-12-09T11:26:49.928 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.gc_set (698 ms) 2019-12-09T11:26:49.928 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.gc_list 2019-12-09T11:26:50.555 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.gc_list (627 ms) 2019-12-09T11:26:50.556 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.gc_defer 2019-12-09T11:26:58.232 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.gc_defer (7677 ms) 2019-12-09T11:26:58.232 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.usage_basic 2019-12-09T11:27:04.829 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.usage_basic (6597 ms) 2019-12-09T11:27:04.829 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-12-09T11:27:04.858 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.usage_clear_no_obj (29 ms) 2019-12-09T11:27:04.858 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.usage_clear 2019-12-09T11:27:13.740 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.usage_clear (8882 ms) 2019-12-09T11:27:13.740 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-12-09T11:27:15.862 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] cls_rgw.bi_log_trim (2123 ms) 2019-12-09T11:27:16.855 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 13 tests from cls_rgw (62619 ms total) 2019-12-09T11:27:16.856 INFO:tasks.workunit.client.0.smithi174.stdout: 2019-12-09T11:27:16.856 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment tear-down 2019-12-09T11:27:16.856 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] 13 tests from 1 test suite ran. (65894 ms total) 2019-12-09T11:27:16.856 INFO:tasks.workunit.client.0.smithi174.stdout:[ PASSED ] 13 tests. 2019-12-09T11:27:16.858 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T11:27:16.858 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T11:27:16.925 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-12-09T11:27:16.926 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-12-09T11:27:16.926 INFO:teuthology.orchestra.run.smithi174:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1351c819e1a9518cde4c8b12d645782009028360 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-12-09T11:27:16.996 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] Running 6 tests from 1 test suite. 2019-12-09T11:27:16.996 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment set-up. 2019-12-09T11:27:19.133 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 6 tests from rgw_gc_log 2019-12-09T11:27:19.134 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-12-09T11:27:19.658 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.init_existing_queue (524 ms) 2019-12-09T11:27:19.658 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-12-09T11:27:19.855 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.init_existing_omap (197 ms) 2019-12-09T11:27:19.855 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-12-09T11:27:19.950 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (95 ms) 2019-12-09T11:27:19.950 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-12-09T11:27:20.003 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (53 ms) 2019-12-09T11:27:20.003 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-12-09T11:27:20.090 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.defer1_after_init (88 ms) 2019-12-09T11:27:20.091 INFO:tasks.workunit.client.0.smithi174.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-12-09T11:27:20.141 INFO:tasks.workunit.client.0.smithi174.stdout:[ OK ] rgw_gc_log.defer2_before_init (51 ms) 2019-12-09T11:27:20.142 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] 6 tests from rgw_gc_log (1008 ms total) 2019-12-09T11:27:20.142 INFO:tasks.workunit.client.0.smithi174.stdout: 2019-12-09T11:27:20.142 INFO:tasks.workunit.client.0.smithi174.stdout:[----------] Global test environment tear-down 2019-12-09T11:27:21.392 INFO:tasks.workunit.client.0.smithi174.stdout:[==========] 6 tests from 1 test suite ran. (4396 ms total) 2019-12-09T11:27:21.393 INFO:tasks.workunit.client.0.smithi174.stdout:[ PASSED ] 6 tests. 2019-12-09T11:27:21.395 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-12-09T11:27:21.395 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-12-09T11:27:21.464 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-12-09T11:27:21.464 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-12-09T11:27:21.716 DEBUG:teuthology.parallel:result is None 2019-12-09T11:27:21.717 INFO:teuthology.orchestra.run.smithi174:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T11:27:21.776 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T11:27:21.776 INFO:teuthology.orchestra.run.smithi174:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-12-09T11:27:21.842 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-12-09T11:27:21.842 INFO:teuthology.run_tasks:Running task ragweed... 2019-12-09T11:27:21.855 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-12-09T11:27:21.855 INFO:tasks.ragweed:Downloading ragweed... 2019-12-09T11:27:21.856 INFO:tasks.ragweed:Using branch 'wip-ofriedma-dynamic-disable' for ragweed 2019-12-09T11:27:21.856 INFO:teuthology.orchestra.run.smithi174:> git clone -b wip-ofriedma-dynamic-disable git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T11:27:21.912 INFO:teuthology.orchestra.run.smithi174.stderr:fatal: Remote branch wip-ofriedma-dynamic-disable not found in upstream origin 2019-12-09T11:27:21.912 INFO:teuthology.orchestra.run.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T11:27:21.912 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-12-09T11:27:21.912 INFO:teuthology.orchestra.run.smithi174:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-12-09T11:27:22.006 INFO:teuthology.orchestra.run.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-12-09T11:27:22.110 INFO:tasks.ragweed:Creating rgw users... 2019-12-09T11:27:22.111 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-12-09T11:27:22.111 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key LEYVWLKEVTWYRUSTTIKG --secret zhzUNbkOYUaxZoa/wjj+Scb4pVbswyyL+FeqvqyeX2AByftf0qCbZg== --email sysuser.client.0+test@test.test --system true 2019-12-09T11:27:22.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.241+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.265+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.265+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.270+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.270+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.270 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.274+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.271 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.274+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.282 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.286+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:22.402 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.406+0000 7f2a45b31ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:27:22.402 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.406+0000 7f2a45b31ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:27:22.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.406+0000 7f2a45b31ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:27:22.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.406+0000 7f2a45b31ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:27:22.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.406+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.408 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.412+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:22.408 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.412+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.420+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:27:22.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.420+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.426+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:22.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.426+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.434+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:27:22.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.434+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.439+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.439+0000 7f2a45b31ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:27:22.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.439+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.443+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.443+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.448 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.451+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:22.448 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.451+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.459+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:27:22.456 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.459+0000 7f2a45b31ec0 20 zonegroup default 2019-12-09T11:27:22.456 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.459+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.464+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.464+0000 7f2a45b31ec0 10 Cannot find current period zone using local zone 2019-12-09T11:27:22.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.464+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.465 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.469+0000 7f2a45b31ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:22.465 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.469+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.473 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.477+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:22.473 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.477+0000 7f2a45b31ec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:22.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.486+0000 7f2a45b31ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:27:22.483 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.486+0000 7f2a45b31ec0 20 zone default found 2019-12-09T11:27:22.483 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.486+0000 7f2a45b31ec0 20 started sync module instance, tier type = 2019-12-09T11:27:22.483 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.486+0000 7f2a45b31ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:27:22.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=0 2019-12-09T11:27:22.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=1 2019-12-09T11:27:22.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=2 2019-12-09T11:27:22.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=3 2019-12-09T11:27:22.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=4 2019-12-09T11:27:22.813 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=5 2019-12-09T11:27:22.813 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.815+0000 7f2a45b31ec0 20 add_watcher() i=6 2019-12-09T11:27:22.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.847+0000 7f2a45b31ec0 20 add_watcher() i=7 2019-12-09T11:27:22.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.847+0000 7f2a45b31ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:27:22.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.847+0000 7f2a037de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:27:22.849 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:22.853+0000 7f2a32d5b700 0 -- 172.21.15.174:0/2940910046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f94d2e24f0 msgr2=0x55f94d2e4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T11:27:23.052 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:23.055+0000 7f2a32d5b700 0 --2- 172.21.15.174:0/2940910046 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55f94d2e24f0 0x55f94d2e4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T11:27:23.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:23.132+0000 7f2a3355c700 0 -- 172.21.15.174:0/2052446425 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f29740224a0 msgr2=0x7f2974024990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:27:24.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.545+0000 7f2971ffb700 20 reqs_thread_entry: start 2019-12-09T11:27:24.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.545+0000 7f296b7fe700 20 reqs_thread_entry: start 2019-12-09T11:27:24.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.546+0000 7f2a45b31ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:27:24.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.546+0000 7f2969ffb700 20 reqs_thread_entry: start 2019-12-09T11:27:24.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.546+0000 7f2a45b31ec0 20 get_system_obj_state: rctx=0x7ffd91f05408 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x55f94d364270 s->prefetch_data=0 2019-12-09T11:27:24.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.547+0000 7f2a45b31ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T11:27:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.565+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-12-09T11:27:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.565+0000 7f2a45b31ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T11:27:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.565+0000 7f2a45b31ec0 20 get_system_obj_state: rctx=0x7ffd91f05418 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x55f94d366000 s->prefetch_data=0 2019-12-09T11:27:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.565+0000 7f2a45b31ec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-12-09T11:27:24.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.579+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-12-09T11:27:24.576 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.579+0000 7f2a45b31ec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T11:27:24.576 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.579+0000 7f2a45b31ec0 20 get_system_obj_state: rctx=0x7ffd91f05418 obj=default.rgw.meta:users.keys:LEYVWLKEVTWYRUSTTIKG state=0x55f94d366000 s->prefetch_data=0 2019-12-09T11:27:24.576 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.579+0000 7f2a45b31ec0 10 cache get: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG : miss 2019-12-09T11:27:24.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.588+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG info.flags=0x0 2019-12-09T11:27:24.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.588+0000 7f2a45b31ec0 10 adding default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG to cache LRU end 2019-12-09T11:27:24.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.588+0000 7f2a45b31ec0 20 get_system_obj_state: rctx=0x7ffd91f05028 obj=default.rgw.meta:users.keys:LEYVWLKEVTWYRUSTTIKG state=0x55f94d366000 s->prefetch_data=0 2019-12-09T11:27:24.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.588+0000 7f2a45b31ec0 10 cache get: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG : type miss (requested=0x6, cached=0x0) 2019-12-09T11:27:24.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.601+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG info.flags=0x0 2019-12-09T11:27:24.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.601+0000 7f2a45b31ec0 10 moving default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG to cache LRU end 2019-12-09T11:27:24.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.601+0000 7f2a45b31ec0 20 get_system_obj_state: rctx=0x7ffd91f051a8 obj=default.rgw.meta:users.keys:LEYVWLKEVTWYRUSTTIKG state=0x55f94d366000 s->prefetch_data=0 2019-12-09T11:27:24.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.601+0000 7f2a45b31ec0 10 cache get: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG : type miss (requested=0x6, cached=0x0) 2019-12-09T11:27:24.607 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.610+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG info.flags=0x0 2019-12-09T11:27:24.607 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.610+0000 7f2a45b31ec0 10 moving default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG to cache LRU end 2019-12-09T11:27:24.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.688+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T11:27:24.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.688+0000 7f2a45b31ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T11:27:24.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.688+0000 7f2a45b31ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-12-09T11:27:24.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.743+0000 7f2a13fff700 10 RGWWatcher::handle_notify() notify_id 541165879296 cookie 94529230012064 notifier 4478 bl.length()=637 2019-12-09T11:27:24.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.743+0000 7f2a13fff700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-12-09T11:27:24.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:24.743+0000 7f2a13fff700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T11:27:25.098 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.102+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T11:27:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.102+0000 7f2a45b31ec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T11:27:25.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.102+0000 7f2a45b31ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-12-09T11:27:25.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.116+0000 7f2a13fff700 10 RGWWatcher::handle_notify() notify_id 541165879297 cookie 94529230016272 notifier 4478 bl.length()=202 2019-12-09T11:27:25.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.116+0000 7f2a13fff700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-12-09T11:27:25.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.116+0000 7f2a13fff700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-12-09T11:27:25.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.405+0000 7f2a45b31ec0 10 cache put: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG info.flags=0x7 2019-12-09T11:27:25.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.405+0000 7f2a45b31ec0 10 moving default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG to cache LRU end 2019-12-09T11:27:25.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.405+0000 7f2a45b31ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=190 2019-12-09T11:27:25.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.443+0000 7f2a13fff700 10 RGWWatcher::handle_notify() notify_id 541165879296 cookie 94529230013760 notifier 4478 bl.length()=190 2019-12-09T11:27:25.440 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.443+0000 7f2a13fff700 10 cache put: name=default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG info.flags=0x7 2019-12-09T11:27:25.440 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.443+0000 7f2a13fff700 10 moving default.rgw.meta+users.keys+LEYVWLKEVTWYRUSTTIKG to cache LRU end 2019-12-09T11:27:25.486 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T11:27:25.486 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "sysuser.client.0", 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. sysuser.client.0", 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "sysuser.client.0+test@test.test", 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T11:27:25.487 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "sysuser.client.0", 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "LEYVWLKEVTWYRUSTTIKG", 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "zhzUNbkOYUaxZoa/wjj+Scb4pVbswyyL+FeqvqyeX2AByftf0qCbZg==" 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T11:27:25.488 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "system": "true", 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T11:27:25.489 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T11:27:25.490 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T11:27:25.491 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T11:27:25.492 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T11:27:25.492 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:25.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.527+0000 7f2a45b31ec0 20 remove_watcher() i=0 2019-12-09T11:27:25.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.527+0000 7f2a45b31ec0 2 removed watcher, disabling cache 2019-12-09T11:27:25.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.594+0000 7f2a45b31ec0 20 remove_watcher() i=1 2019-12-09T11:27:25.646 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.650+0000 7f2a45b31ec0 20 remove_watcher() i=2 2019-12-09T11:27:25.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.716+0000 7f2a45b31ec0 20 remove_watcher() i=3 2019-12-09T11:27:25.781 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.784+0000 7f2a45b31ec0 20 remove_watcher() i=4 2019-12-09T11:27:25.830 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.834+0000 7f2a45b31ec0 20 remove_watcher() i=5 2019-12-09T11:27:25.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.895+0000 7f2a45b31ec0 20 remove_watcher() i=6 2019-12-09T11:27:25.938 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:25.942+0000 7f2a45b31ec0 20 remove_watcher() i=7 2019-12-09T11:27:25.945 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-12-09T11:27:25.945 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key NZYBRWCJCEYDZQXBSMTK --secret eqjWAg+snhp7X2MULaQdEgvgczKKz1vVGAax3ylC+p545Mrdv361IA== --email ragweed.client.0+test@test.test --system false 2019-12-09T11:27:26.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.103+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.132+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.132+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.137 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.141+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.137 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.141+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.144 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.148+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.148+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.163 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.167+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:26.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.365+0000 7f415ae8aec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:27:26.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.365+0000 7f415ae8aec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:27:26.362 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.365+0000 7f415ae8aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:27:26.362 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.365+0000 7f415ae8aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:27:26.362 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.365+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.386+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:26.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.386+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.398+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:27:26.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.398+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.418+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:26.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.418+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.433+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:27:26.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.433+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.439+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.439+0000 7f415ae8aec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:27:26.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.439+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.451+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.447 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.451+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.462 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.466+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:26.463 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.466+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.476+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:27:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.476+0000 7f415ae8aec0 20 zonegroup default 2019-12-09T11:27:26.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.476+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.484+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.484+0000 7f415ae8aec0 10 Cannot find current period zone using local zone 2019-12-09T11:27:26.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.484+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.492+0000 7f415ae8aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:27:26.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.492+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.513+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:27:26.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.513+0000 7f415ae8aec0 20 rados->read ofs=0 len=0 2019-12-09T11:27:26.518 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.521+0000 7f415ae8aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:27:26.518 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.521+0000 7f415ae8aec0 20 zone default found 2019-12-09T11:27:26.518 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.521+0000 7f415ae8aec0 20 started sync module instance, tier type = 2019-12-09T11:27:26.518 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.521+0000 7f415ae8aec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:27:26.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=0 2019-12-09T11:27:26.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=1 2019-12-09T11:27:26.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=2 2019-12-09T11:27:26.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=3 2019-12-09T11:27:26.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=4 2019-12-09T11:27:26.878 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.879+0000 7f415ae8aec0 20 add_watcher() i=5 2019-12-09T11:27:26.878 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.881+0000 7f415ae8aec0 20 add_watcher() i=6 2019-12-09T11:27:26.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.913+0000 7f415ae8aec0 20 add_watcher() i=7 2019-12-09T11:27:26.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.913+0000 7f415ae8aec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:27:26.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.913+0000 7f4118fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:27:26.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.914+0000 7f41478b3700 0 -- >> v1:172.21.15.174:6789/0 conn(0x5556b47be040 legacy=0x5556b47c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T11:27:26.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:26.951+0000 7f41488b5700 0 -- 172.21.15.174:0/2316554522 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f408c022470 msgr2=0x7f408c024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:27:28.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.384+0000 7f408affd700 20 reqs_thread_entry: start 2019-12-09T11:27:28.381 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.384+0000 7f4088ff9700 20 reqs_thread_entry: start 2019-12-09T11:27:28.381 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.384+0000 7f415ae8aec0 20 init_complete bucket index max shards: 0 2019-12-09T11:27:28.381 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.385+0000 7f407effd700 20 reqs_thread_entry: start 2019-12-09T11:27:28.382 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.385+0000 7f415ae8aec0 20 get_system_obj_state: rctx=0x7ffeb969b038 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x5556b483b050 s->prefetch_data=0 2019-12-09T11:27:28.382 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.385+0000 7f415ae8aec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T11:27:28.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.393+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-12-09T11:27:28.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.393+0000 7f415ae8aec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T11:27:28.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.393+0000 7f415ae8aec0 20 get_system_obj_state: rctx=0x7ffeb969b048 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x5556b483cde0 s->prefetch_data=0 2019-12-09T11:27:28.390 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.393+0000 7f415ae8aec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-12-09T11:27:28.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.399+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-12-09T11:27:28.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.399+0000 7f415ae8aec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T11:27:28.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.399+0000 7f415ae8aec0 20 get_system_obj_state: rctx=0x7ffeb969b048 obj=default.rgw.meta:users.keys:NZYBRWCJCEYDZQXBSMTK state=0x5556b483cde0 s->prefetch_data=0 2019-12-09T11:27:28.396 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.399+0000 7f415ae8aec0 10 cache get: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK : miss 2019-12-09T11:27:28.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.410+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK info.flags=0x0 2019-12-09T11:27:28.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.410+0000 7f415ae8aec0 10 adding default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK to cache LRU end 2019-12-09T11:27:28.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.410+0000 7f415ae8aec0 20 get_system_obj_state: rctx=0x7ffeb969ac58 obj=default.rgw.meta:users.keys:NZYBRWCJCEYDZQXBSMTK state=0x5556b483cde0 s->prefetch_data=0 2019-12-09T11:27:28.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.410+0000 7f415ae8aec0 10 cache get: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK : type miss (requested=0x6, cached=0x0) 2019-12-09T11:27:28.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.419+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK info.flags=0x0 2019-12-09T11:27:28.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.419+0000 7f415ae8aec0 10 moving default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK to cache LRU end 2019-12-09T11:27:28.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.419+0000 7f415ae8aec0 20 get_system_obj_state: rctx=0x7ffeb969add8 obj=default.rgw.meta:users.keys:NZYBRWCJCEYDZQXBSMTK state=0x5556b483cde0 s->prefetch_data=0 2019-12-09T11:27:28.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.419+0000 7f415ae8aec0 10 cache get: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK : type miss (requested=0x6, cached=0x0) 2019-12-09T11:27:28.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.428+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK info.flags=0x0 2019-12-09T11:27:28.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.428+0000 7f415ae8aec0 10 moving default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK to cache LRU end 2019-12-09T11:27:28.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.487+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T11:27:28.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.487+0000 7f415ae8aec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T11:27:28.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.487+0000 7f415ae8aec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-12-09T11:27:28.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.498+0000 7f412dffb700 10 RGWWatcher::handle_notify() notify_id 541165879298 cookie 93830883484384 notifier 4490 bl.length()=637 2019-12-09T11:27:28.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.498+0000 7f412dffb700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-12-09T11:27:28.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.498+0000 7f412dffb700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T11:27:28.701 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.705+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T11:27:28.702 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.705+0000 7f415ae8aec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T11:27:28.702 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.705+0000 7f415ae8aec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-12-09T11:27:28.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.728+0000 7f412dffb700 10 RGWWatcher::handle_notify() notify_id 541165879299 cookie 93830883479712 notifier 4490 bl.length()=202 2019-12-09T11:27:28.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.728+0000 7f412dffb700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-12-09T11:27:28.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:28.728+0000 7f412dffb700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-12-09T11:27:29.108 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.112+0000 7f415ae8aec0 10 cache put: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK info.flags=0x7 2019-12-09T11:27:29.108 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.112+0000 7f415ae8aec0 10 moving default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK to cache LRU end 2019-12-09T11:27:29.109 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.112+0000 7f415ae8aec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=190 2019-12-09T11:27:29.170 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.173+0000 7f412dffb700 10 RGWWatcher::handle_notify() notify_id 541165879296 cookie 93830883486336 notifier 4490 bl.length()=190 2019-12-09T11:27:29.170 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.173+0000 7f412dffb700 10 cache put: name=default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK info.flags=0x7 2019-12-09T11:27:29.170 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.173+0000 7f412dffb700 10 moving default.rgw.meta+users.keys+NZYBRWCJCEYDZQXBSMTK to cache LRU end 2019-12-09T11:27:29.475 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T11:27:29.476 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "ragweed.client.0", 2019-12-09T11:27:29.476 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. ragweed.client.0", 2019-12-09T11:27:29.476 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "ragweed.client.0+test@test.test", 2019-12-09T11:27:29.476 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T11:27:29.476 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "ragweed.client.0", 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "NZYBRWCJCEYDZQXBSMTK", 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "eqjWAg+snhp7X2MULaQdEgvgczKKz1vVGAax3ylC+p545Mrdv361IA==" 2019-12-09T11:27:29.477 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "system": "true", 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T11:27:29.478 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:27:29.479 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:27:29.480 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T11:27:29.481 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:29.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.529+0000 7f415ae8aec0 20 remove_watcher() i=0 2019-12-09T11:27:29.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.529+0000 7f415ae8aec0 2 removed watcher, disabling cache 2019-12-09T11:27:29.571 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.575+0000 7f415ae8aec0 20 remove_watcher() i=1 2019-12-09T11:27:29.623 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.627+0000 7f415ae8aec0 20 remove_watcher() i=2 2019-12-09T11:27:29.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.666+0000 7f415ae8aec0 20 remove_watcher() i=3 2019-12-09T11:27:29.723 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.727+0000 7f415ae8aec0 20 remove_watcher() i=4 2019-12-09T11:27:29.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.766+0000 7f415ae8aec0 20 remove_watcher() i=5 2019-12-09T11:27:29.802 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.806+0000 7f415ae8aec0 20 remove_watcher() i=6 2019-12-09T11:27:29.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:27:29.837+0000 7f415ae8aec0 20 remove_watcher() i=7 2019-12-09T11:27:29.841 INFO:tasks.ragweed:Configuring ragweed... 2019-12-09T11:27:29.841 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-12-09T11:27:36.322 INFO:teuthology.orchestra.run.smithi174.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T11:27:36.425 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T11:27:36.460 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T11:27:36.461 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T11:27:36.462 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T11:27:36.462 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T11:27:36.462 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T11:27:37.077 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T11:27:37.427 INFO:teuthology.orchestra.run.smithi174.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-12-09T11:27:38.312 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T11:27:38.313 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:27:38.313 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-12-09T11:27:38.315 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-12-09T11:27:38.317 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-12-09T11:27:38.351 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-12-09T11:27:38.352 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-12-09T11:27:38.354 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-12-09T11:27:38.355 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:27:38.355 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-12-09T11:27:38.357 INFO:teuthology.orchestra.run.smithi174.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-12-09T11:27:38.360 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-12-09T11:27:38.360 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T11:27:38.361 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-12-09T11:27:38.767 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T11:27:38.962 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:38.963 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T11:27:38.964 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:38.964 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:27:38.964 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-12-09T11:27:38.965 INFO:teuthology.orchestra.run.smithi174.stdout:Installing for dependencies: 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T11:27:38.966 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages (+3 Dependent packages) 2019-12-09T11:27:38.967 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:38.967 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 1.7 M 2019-12-09T11:27:38.967 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 12 M 2019-12-09T11:27:38.967 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T11:27:39.667 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2019-12-09T11:27:39.667 INFO:teuthology.orchestra.run.smithi174.stdout:Total 2.4 MB/s | 1.7 MB 00:00 2019-12-09T11:27:39.675 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T11:27:39.690 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T11:27:39.764 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T11:27:39.765 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T11:27:40.470 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-12-09T11:27:40.860 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-12-09T11:27:41.124 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T11:27:41.654 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-12-09T11:27:42.120 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-12-09T11:27:42.571 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-12-09T11:27:42.873 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-12-09T11:27:43.031 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-12-09T11:27:43.149 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-12-09T11:27:43.224 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-12-09T11:27:43.341 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-12-09T11:27:43.434 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-12-09T11:27:43.559 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-12-09T11:27:44.018 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-12-09T11:27:44.018 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout:Dependency Installed: 2019-12-09T11:27:44.019 INFO:teuthology.orchestra.run.smithi174.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-12-09T11:27:44.020 INFO:teuthology.orchestra.run.smithi174.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-12-09T11:27:44.020 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:44.020 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T11:27:44.271 INFO:teuthology.orchestra.run.smithi174.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-12-09T11:27:45.825 INFO:teuthology.orchestra.run.smithi174.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T11:27:46.294 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting pip 2019-12-09T11:27:46.420 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-12-09T11:27:46.619 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: pip 2019-12-09T11:27:46.619 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: pip 9.0.1 2019-12-09T11:27:46.681 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling pip-9.0.1: 2019-12-09T11:27:46.722 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T11:27:47.436 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed pip-19.3.1 2019-12-09T11:27:48.232 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T11:27:48.321 INFO:teuthology.orchestra.run.smithi174.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-12-09T11:27:49.168 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T11:27:49.387 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting nose 2019-12-09T11:27:49.479 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-12-09T11:27:49.644 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: nose 2019-12-09T11:27:49.715 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed nose-1.3.7 2019-12-09T11:27:50.256 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T11:27:50.883 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting setuptools==32.3.1 2019-12-09T11:27:50.930 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-12-09T11:27:51.142 INFO:teuthology.orchestra.run.smithi174.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T11:27:51.142 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: setuptools 2019-12-09T11:27:51.142 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T11:27:51.167 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T11:27:51.180 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T11:27:51.315 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed setuptools-32.3.1 2019-12-09T11:27:51.846 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T11:27:52.248 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting PyYAML 2019-12-09T11:27:52.298 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB) 2019-12-09T11:27:53.002 INFO:teuthology.orchestra.run.smithi174.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-12-09T11:27:53.155 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-12-09T11:27:53.168 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-12-09T11:27:53.442 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting bunch>=1.0.0 2019-12-09T11:27:53.452 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-12-09T11:27:54.144 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting gevent>=1.0 2019-12-09T11:27:54.164 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-12-09T11:27:54.618 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting httplib2 2019-12-09T11:27:54.627 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-12-09T11:27:55.869 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting lxml 2019-12-09T11:27:55.880 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-12-09T11:27:56.412 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T11:27:56.422 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T11:27:56.501 INFO:teuthology.orchestra.run.smithi174.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-12-09T11:27:56.502 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for PyYAML (setup.py): started 2019-12-09T11:27:57.340 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-12-09T11:27:57.341 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for PyYAML: filename=PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl size=46299 sha256=26ba9433c7b95a23ef06791f322ce1831add0206ae171aaf9a78ac5ffdb8e34b 2019-12-09T11:27:57.341 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67 2019-12-09T11:27:57.363 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for bunch (setup.py): started 2019-12-09T11:27:57.685 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-12-09T11:27:57.686 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=a180369c43c551676881f70c33fb4dab9a58cb72479f426d9fa09f9021dc3713 2019-12-09T11:27:57.686 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-12-09T11:27:57.690 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for httplib2 (setup.py): started 2019-12-09T11:27:58.046 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-12-09T11:27:58.047 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=30f30f9d471591418c79d65718b0917ed6dc69464fc6c0852f1ac0fb99b903da 2019-12-09T11:27:58.047 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-12-09T11:27:58.052 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully built PyYAML bunch httplib2 2019-12-09T11:27:58.158 INFO:teuthology.orchestra.run.smithi174.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-12-09T11:27:58.158 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-12-09T11:27:58.158 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: PyYAML 3.10 2019-12-09T11:27:58.159 INFO:teuthology.orchestra.run.smithi174.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T11:27:58.159 INFO:teuthology.orchestra.run.smithi174.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-12-09T11:27:59.225 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: lxml 3.2.1 2019-12-09T11:27:59.226 INFO:teuthology.orchestra.run.smithi174.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-12-09T11:27:59.226 INFO:teuthology.orchestra.run.smithi174.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-12-09T11:27:59.303 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.2 2019-12-09T11:27:59.732 INFO:teuthology.orchestra.run.smithi174.stdout:running develop 2019-12-09T11:27:59.798 INFO:teuthology.orchestra.run.smithi174.stdout:running egg_info 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:creating ragweed.egg-info 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:writing ragweed.egg-info/PKG-INFO 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-12-09T11:27:59.799 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T11:27:59.800 INFO:teuthology.orchestra.run.smithi174.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T11:27:59.800 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-12-09T11:27:59.801 INFO:teuthology.orchestra.run.smithi174.stdout:running build_ext 2019-12-09T11:27:59.801 INFO:teuthology.orchestra.run.smithi174.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-12-09T11:27:59.801 INFO:teuthology.orchestra.run.smithi174.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-12-09T11:27:59.802 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:27:59.802 INFO:teuthology.orchestra.run.smithi174.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-12-09T11:27:59.803 INFO:teuthology.orchestra.run.smithi174.stdout:Processing dependencies for ragweed==0.0.1 2019-12-09T11:27:59.807 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for isodate>=0.4.4 2019-12-09T11:27:59.807 INFO:teuthology.orchestra.run.smithi174.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-12-09T11:27:59.908 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-12-09T11:27:59.972 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: isodate 0.6.0 2019-12-09T11:27:59.972 INFO:teuthology.orchestra.run.smithi174.stdout:Processing isodate-0.6.0.tar.gz 2019-12-09T11:27:59.986 INFO:teuthology.orchestra.run.smithi174.stdout:Writing /tmp/easy_install-2qBoYg/isodate-0.6.0/setup.cfg 2019-12-09T11:27:59.986 INFO:teuthology.orchestra.run.smithi174.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-2qBoYg/isodate-0.6.0/egg-dist-tmp-NabI0z 2019-12-09T11:28:00.295 INFO:teuthology.orchestra.run.smithi174.stderr:zip_safe flag not set; analyzing archive contents... 2019-12-09T11:28:00.309 INFO:teuthology.orchestra.run.smithi174.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.311 INFO:teuthology.orchestra.run.smithi174.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T11:28:00.312 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.312 INFO:teuthology.orchestra.run.smithi174.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-12-09T11:28:00.335 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for gevent==1.4.0 2019-12-09T11:28:00.335 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: gevent 1.4.0 2019-12-09T11:28:00.335 INFO:teuthology.orchestra.run.smithi174.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T11:28:00.336 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.336 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.337 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for bunch==1.0.1 2019-12-09T11:28:00.337 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: bunch 1.0.1 2019-12-09T11:28:00.337 INFO:teuthology.orchestra.run.smithi174.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T11:28:00.337 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.337 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.338 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for PyYAML==5.2 2019-12-09T11:28:00.338 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: PyYAML 5.2 2019-12-09T11:28:00.338 INFO:teuthology.orchestra.run.smithi174.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for boto==2.49.0 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: boto 2.49.0 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.339 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.340 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for six==1.9.0 2019-12-09T11:28:00.340 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: six 1.9.0 2019-12-09T11:28:00.340 INFO:teuthology.orchestra.run.smithi174.stdout:Adding six 1.9.0 to easy-install.pth file 2019-12-09T11:28:00.341 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.341 INFO:teuthology.orchestra.run.smithi174.stdout:Using /usr/lib/python2.7/site-packages 2019-12-09T11:28:00.349 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for greenlet==0.4.15 2019-12-09T11:28:00.349 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: greenlet 0.4.15 2019-12-09T11:28:00.349 INFO:teuthology.orchestra.run.smithi174.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T11:28:00.350 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:28:00.350 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-12-09T11:28:00.350 INFO:teuthology.orchestra.run.smithi174.stdout:Finished processing dependencies for ragweed==0.0.1 2019-12-09T11:28:00.375 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-12-09T11:28:00.439 INFO:tasks.ragweed:Configuring boto... 2019-12-09T11:28:00.440 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T11:28:00.528 INFO:teuthology.orchestra.run.smithi174:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-12-09T11:28:45.316 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-12-09T11:29:19.331 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-12-09T11:29:49.667 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-12-09T11:30:10.487 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:09:02.323 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:10.569 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:02.401 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:10.574 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:09:02.410 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:10.654 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:02.487 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:10.660 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:02.492 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:10.730 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:02.563 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:11.200 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:09:03.037 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:11.255 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:09:03.088 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:11.350 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:09:03.182 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:30:23.312 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-12-09T11:31:07.657 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-12-09T11:32:28.598 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:11:20.438 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.608 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:11:20.448 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.619 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:20.453 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.621 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:20.456 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.625 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:20.459 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.625 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:20.460 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.755 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:11:20.589 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.783 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:11:20.589 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:28.783 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:11:20.612 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:32:49.575 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-12-09T11:34:05.232 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-12-09T11:37:04.508 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-12-09T11:37:41.621 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:16:33.467 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.625 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:16:33.471 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.664 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:33.501 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.683 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:33.520 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.686 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:33.524 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.690 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:33.528 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.850 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:16:33.696 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.867 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:16:33.705 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:41.870 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:16:33.708 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:37:53.325 INFO:teuthology.orchestra.run.smithi174.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-12-09T11:37:53.326 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T11:37:53.327 INFO:teuthology.orchestra.run.smithi174.stderr:---------------------------------------------------------------------- 2019-12-09T11:37:53.327 INFO:teuthology.orchestra.run.smithi174.stderr:Ran 9 tests in 592.389s 2019-12-09T11:37:53.327 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T11:37:53.327 INFO:teuthology.orchestra.run.smithi174.stderr:OK 2019-12-09T11:37:53.351 INFO:tasks.ragweed:Cleaning up boto... 2019-12-09T11:37:53.351 INFO:teuthology.orchestra.run.smithi174:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T11:37:53.420 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-12-09T11:37:53.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.546+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.579+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.579+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.589+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.589+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.601+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.601+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.606 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.617+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:53.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.806+0000 7fbbda41dec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:37:53.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.806+0000 7fbbda41dec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:37:53.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.806+0000 7fbbda41dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:37:53.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.806+0000 7fbbda41dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:37:53.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.806+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.822+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:53.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.822+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.837+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:37:53.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.837+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.854+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:53.843 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.854+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.863+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:37:53.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.863+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.873+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.873+0000 7fbbda41dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:37:53.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.873+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.871 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.881+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.871 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.881+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.885 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.895+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:53.885 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.895+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.909+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:37:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.909+0000 7fbbda41dec0 20 zonegroup default 2019-12-09T11:37:53.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.909+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.919+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.919+0000 7fbbda41dec0 10 Cannot find current period zone using local zone 2019-12-09T11:37:53.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.919+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.928+0000 7fbbda41dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:53.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.928+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.940+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:53.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.940+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:53.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.951+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:37:53.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.951+0000 7fbbda41dec0 20 zone default found 2019-12-09T11:37:53.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.951+0000 7fbbda41dec0 20 started sync module instance, tier type = 2019-12-09T11:37:53.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:53.951+0000 7fbbda41dec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:37:54.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=0 2019-12-09T11:37:54.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=1 2019-12-09T11:37:54.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=2 2019-12-09T11:37:54.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=3 2019-12-09T11:37:54.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=4 2019-12-09T11:37:54.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=5 2019-12-09T11:37:54.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.303+0000 7fbbda41dec0 20 add_watcher() i=6 2019-12-09T11:37:54.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.331+0000 7fbbda41dec0 20 add_watcher() i=7 2019-12-09T11:37:54.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.331+0000 7fbbda41dec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:37:54.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.331+0000 7fbb97fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:37:54.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.344+0000 7fbbc6e46700 0 -- 172.21.15.174:0/2368729969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5650b5f96330 msgr2=0x5650b5f98840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:37:54.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:54.366+0000 7fbbc7e48700 0 -- 172.21.15.174:0/1950055654 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5650b5f94ef0 msgr2=0x5650b5f95300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:37:56.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.077+0000 7fbb067fc700 20 reqs_thread_entry: start 2019-12-09T11:37:56.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.078+0000 7fbafffff700 20 reqs_thread_entry: start 2019-12-09T11:37:56.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.078+0000 7fbbda41dec0 20 init_complete bucket index max shards: 0 2019-12-09T11:37:56.068 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.078+0000 7fbafe7fc700 20 reqs_thread_entry: start 2019-12-09T11:37:56.068 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.078+0000 7fbbda41dec0 20 get_system_obj_state: rctx=0x7fff400b4588 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x5650b600e2c0 s->prefetch_data=0 2019-12-09T11:37:56.068 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.078+0000 7fbbda41dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-12-09T11:37:56.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-12-09T11:37:56.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T11:37:56.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:37:56.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:37:56.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T11:37:56.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.089+0000 7fbbda41dec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:56.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.102+0000 7fbbda41dec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T11:37:56.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.102+0000 7fbbda41dec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-12-09T11:37:56.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.102+0000 7fbbda41dec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-12-09T11:37:56.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.109+0000 7fbbda41dec0 10 removing key index: LEYVWLKEVTWYRUSTTIKG 2019-12-09T11:37:56.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.109+0000 7fbbda41dec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=170 2019-12-09T11:37:56.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.128+0000 7fbbbcff9700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94904650302608 notifier 4508 bl.length()=170 2019-12-09T11:37:56.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.319+0000 7fbbda41dec0 10 removing email index: sysuser.client.0+test@test.test 2019-12-09T11:37:56.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.319+0000 7fbbda41dec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-12-09T11:37:56.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.334+0000 7fbbbcff9700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94904650306192 notifier 4508 bl.length()=182 2019-12-09T11:37:56.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.789+0000 7fbbda41dec0 10 removing user buckets index 2019-12-09T11:37:56.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.789+0000 7fbbda41dec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-12-09T11:37:56.794 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:56.805+0000 7fbbbcff9700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94904650302608 notifier 4508 bl.length()=173 2019-12-09T11:37:57.552 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:57.563+0000 7fbbda41dec0 10 removing user index: sysuser.client.0 2019-12-09T11:37:57.553 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:57.563+0000 7fbbda41dec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-12-09T11:37:57.553 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:57.563+0000 7fbbda41dec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-12-09T11:37:57.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:57.585+0000 7fbbbcff9700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94904650300976 notifier 4508 bl.length()=165 2019-12-09T11:37:58.054 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.064+0000 7fbbda41dec0 20 remove_watcher() i=0 2019-12-09T11:37:58.054 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.064+0000 7fbbda41dec0 2 removed watcher, disabling cache 2019-12-09T11:37:58.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.137+0000 7fbbda41dec0 20 remove_watcher() i=1 2019-12-09T11:37:58.187 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.198+0000 7fbbda41dec0 20 remove_watcher() i=2 2019-12-09T11:37:58.241 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.251+0000 7fbbda41dec0 20 remove_watcher() i=3 2019-12-09T11:37:58.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.300+0000 7fbbda41dec0 20 remove_watcher() i=4 2019-12-09T11:37:58.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.344+0000 7fbbda41dec0 20 remove_watcher() i=5 2019-12-09T11:37:58.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.405+0000 7fbbda41dec0 20 remove_watcher() i=6 2019-12-09T11:37:58.446 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.456+0000 7fbbda41dec0 20 remove_watcher() i=7 2019-12-09T11:37:58.455 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-12-09T11:37:58.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.612+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.640+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.640+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.638 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.649+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.638 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.649+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.655+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.655+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.674+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:58.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.831+0000 7fd8f5d20ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:37:58.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.831+0000 7fd8f5d20ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:37:58.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.831+0000 7fd8f5d20ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:37:58.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.831+0000 7fd8f5d20ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:37:58.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.831+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.837 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.848+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:58.837 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.848+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.860+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:37:58.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.860+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.871+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:58.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.871+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.872 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.883+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:37:58.872 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.883+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.893+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.893+0000 7fd8f5d20ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:37:58.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.893+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.901+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.901+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.905 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.915+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:58.905 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.915+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.924+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:37:58.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.924+0000 7fd8f5d20ec0 20 zonegroup default 2019-12-09T11:37:58.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.924+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.931+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.931+0000 7fd8f5d20ec0 10 Cannot find current period zone using local zone 2019-12-09T11:37:58.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.931+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.940+0000 7fd8f5d20ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:37:58.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.940+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.949+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:37:58.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.949+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:37:58.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.960+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:37:58.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.960+0000 7fd8f5d20ec0 20 zone default found 2019-12-09T11:37:58.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.960+0000 7fd8f5d20ec0 20 started sync module instance, tier type = 2019-12-09T11:37:58.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:58.960+0000 7fd8f5d20ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:37:59.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8e2f4a700 0 -- 172.21.15.174:0/4234148934 >> [v2:172.21.15.174:6802/10741,v1:172.21.15.174:6803/10741] conn(0x7fd8cc0223e0 msgr2=0x7fd8cc024040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:37:59.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=0 2019-12-09T11:37:59.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=1 2019-12-09T11:37:59.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=2 2019-12-09T11:37:59.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=3 2019-12-09T11:37:59.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=4 2019-12-09T11:37:59.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.331+0000 7fd8f5d20ec0 20 add_watcher() i=5 2019-12-09T11:37:59.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.334+0000 7fd8f5d20ec0 20 add_watcher() i=6 2019-12-09T11:37:59.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.398+0000 7fd8f5d20ec0 20 add_watcher() i=7 2019-12-09T11:37:59.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.398+0000 7fd8f5d20ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:37:59.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.398+0000 7fd8b37de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:37:59.404 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:37:59.415+0000 7fd8e374b700 0 -- 172.21.15.174:0/206595080 >> v2:172.21.15.106:3300/0 conn(0x55a187e95fb0 msgr2=0x55a187e963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:38:00.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.848+0000 7fd821ffb700 20 reqs_thread_entry: start 2019-12-09T11:38:00.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.848+0000 7fd81b7fe700 20 reqs_thread_entry: start 2019-12-09T11:38:00.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.848+0000 7fd8f5d20ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:38:00.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.849+0000 7fd819ffb700 20 reqs_thread_entry: start 2019-12-09T11:38:00.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.849+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90688 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55a187f11de0 s->prefetch_data=0 2019-12-09T11:38:00.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.849+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-12-09T11:38:00.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-12-09T11:38:00.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T11:38:00.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:00.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:00.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T11:38:00.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.866+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:00.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.893+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=448 2019-12-09T11:38:00.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.894+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-12-09T11:38:00.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.894+0000 7fd8f5d20ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-12-09T11:38:00.902 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.912+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-conf state=0x55a187f13700 s->prefetch_data=0 2019-12-09T11:38:00.902 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.912+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-12-09T11:38:00.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-12-09T11:38:00.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T11:38:00.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:00.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:00.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:00.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.930+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:00.932 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.942+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=200 2019-12-09T11:38:00.932 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.942+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-12-09T11:38:00.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.943+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T11:38:00.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.943+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 state=0x55a187f15cc0 s->prefetch_data=0 2019-12-09T11:38:00.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.943+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 : miss 2019-12-09T11:38:00.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 info.flags=0x16 2019-12-09T11:38:00.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 to cache LRU end 2019-12-09T11:38:00.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:00.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:00.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:00.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:00.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:00.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:00.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.966+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:00.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=292 2019-12-09T11:38:00.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 info.flags=0x13 2019-12-09T11:38:00.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 to cache LRU end 2019-12-09T11:38:00.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:00.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:00.987 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:00.996+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:01.009 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.020+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-conf[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1]) start [] num_entries 1100 2019-12-09T11:38:01.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-12-09T11:38:01.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-12-09T11:38:01.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-12-09T11:38:01.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-12-09T11:38:01.040 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.048+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.067+0000 7fd8f5d20ec0 10 manifest: total_size = 5933 2019-12-09T11:38:01.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.067+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.13 2019-12-09T11:38:01.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.067+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.067+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:01.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.067+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:01.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.314+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.331+0000 7fd8f5d20ec0 10 manifest: total_size = 5858 2019-12-09T11:38:01.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.331+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.30 2019-12-09T11:38:01.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.331+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.331+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:01.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.331+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:01.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.540+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.551+0000 7fd8f5d20ec0 10 manifest: total_size = 3807 2019-12-09T11:38:01.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.551+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.56 2019-12-09T11:38:01.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.551+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.551+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:01.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.551+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:01.708 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.719+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.723 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.733+0000 7fd8f5d20ec0 10 manifest: total_size = 3793 2019-12-09T11:38:01.723 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.733+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.72 2019-12-09T11:38:01.723 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.733+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.733+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:01.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.733+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:01.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.889+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.898 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.908+0000 7fd8f5d20ec0 10 manifest: total_size = 5960 2019-12-09T11:38:01.898 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.908+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.87 2019-12-09T11:38:01.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.908+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:01.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.908+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:01.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:01.908+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:02.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.069+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.091 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.102+0000 7fd8f5d20ec0 10 manifest: total_size = 6055 2019-12-09T11:38:02.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.102+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.120 2019-12-09T11:38:02.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.102+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.102+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:02.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.102+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:02.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.252+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.292 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.302+0000 7fd8f5d20ec0 10 manifest: total_size = 6195 2019-12-09T11:38:02.306 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.302+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.189 2019-12-09T11:38:02.306 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.302+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.306 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.302+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:02.306 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.302+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:02.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.482+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.495+0000 7fd8f5d20ec0 10 manifest: total_size = 12953 2019-12-09T11:38:02.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.495+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.246 2019-12-09T11:38:02.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.495+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.495+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:02.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.495+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:02.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.737+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.753 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.763+0000 7fd8f5d20ec0 10 manifest: total_size = 4001 2019-12-09T11:38:02.754 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.763+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.322 2019-12-09T11:38:02.754 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.763+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x55a187f1bc58 s->prefetch_data=0 2019-12-09T11:38:02.754 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.763+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1bc58 2019-12-09T11:38:02.754 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.763+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 2019-12-09T11:38:02.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.947+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-conf[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1]) start _multipart_[] num_entries 1001 2019-12-09T11:38:02.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:02.947+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:02.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.009+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:03.060 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.070+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-conf state=0x55a187f22530 s->prefetch_data=0 2019-12-09T11:38:03.060 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.070+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:03.070 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-12-09T11:38:03.070 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-12-09T11:38:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.080+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-12-09T11:38:03.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.081+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-12-09T11:38:03.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.109+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 94152258222928 notifier 4517 bl.length()=162 2019-12-09T11:38:03.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.488+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.1 from cache 2019-12-09T11:38:03.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.488+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=219 2019-12-09T11:38:03.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.501+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94152258224992 notifier 4517 bl.length()=219 2019-12-09T11:38:03.832 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.841+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55a187f1ab60 s->prefetch_data=0 2019-12-09T11:38:03.832 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.841+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-12-09T11:38:03.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-12-09T11:38:03.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T11:38:03.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:03.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:03.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:03.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.851+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:03.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.864+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=229 2019-12-09T11:38:03.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.864+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-12-09T11:38:03.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.864+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T11:38:03.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.864+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:03.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.864+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 : miss 2019-12-09T11:38:03.864 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.873+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 info.flags=0x16 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.873+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 to cache LRU end 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:03.866 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:03.866 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.874+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:03.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=321 2019-12-09T11:38:03.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 info.flags=0x13 2019-12-09T11:38:03.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 to cache LRU end 2019-12-09T11:38:03.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:03.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:03.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.901+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 2019-12-09T11:38:03.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.919+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2]) start [] num_entries 1100 2019-12-09T11:38:03.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.932+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T11:38:03.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.932+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55a187dc76e8 s->prefetch_data=0 2019-12-09T11:38:03.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.944+0000 7fd8f5d20ec0 10 manifest: total_size = 1848 2019-12-09T11:38:03.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.944+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.20 2019-12-09T11:38:03.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.944+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x55a187dc76e8 s->prefetch_data=0 2019-12-09T11:38:03.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.944+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc76e8 2019-12-09T11:38:03.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:03.944+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 2019-12-09T11:38:04.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-12-09T11:38:04.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-12-09T11:38:04.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-12-09T11:38:04.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-12-09T11:38:04.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-12-09T11:38:04.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-12-09T11:38:04.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.123+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.20 2019-12-09T11:38:04.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.222+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2]) start _multipart_[] num_entries 1001 2019-12-09T11:38:04.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.222+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:04.227 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.236+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:04.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.286+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x55a187f19080 s->prefetch_data=0 2019-12-09T11:38:04.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.287+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-12-09T11:38:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-12-09T11:38:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:04.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:04.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-12-09T11:38:04.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.297+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-12-09T11:38:04.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.314+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 94152258224992 notifier 4517 bl.length()=191 2019-12-09T11:38:04.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.850+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2 from cache 2019-12-09T11:38:04.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.850+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=248 2019-12-09T11:38:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:04.866+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94152258226480 notifier 4517 bl.length()=248 2019-12-09T11:38:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.183+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:05.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.183+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-12-09T11:38:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-12-09T11:38:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T11:38:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:05.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:05.187 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.195+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:05.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.210+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=236 2019-12-09T11:38:05.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.210+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-12-09T11:38:05.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.210+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T11:38:05.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.211+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 state=0x55a187f22530 s->prefetch_data=0 2019-12-09T11:38:05.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.211+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 : miss 2019-12-09T11:38:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 info.flags=0x16 2019-12-09T11:38:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 to cache LRU end 2019-12-09T11:38:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:05.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:05.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.228+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:05.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.244+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=328 2019-12-09T11:38:05.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.245+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 info.flags=0x13 2019-12-09T11:38:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.245+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 to cache LRU end 2019-12-09T11:38:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.245+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.245+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:05.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.245+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 2019-12-09T11:38:05.246 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.256+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3]) start [] num_entries 1100 2019-12-09T11:38:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.269+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T11:38:05.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.269+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55a187dc76e8 s->prefetch_data=0 2019-12-09T11:38:05.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.286+0000 7fd8f5d20ec0 10 manifest: total_size = 74198 2019-12-09T11:38:05.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.286+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.40 2019-12-09T11:38:05.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.286+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x55a187dc76e8 s->prefetch_data=0 2019-12-09T11:38:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.286+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc76e8 2019-12-09T11:38:05.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.286+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 2019-12-09T11:38:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T11:38:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T11:38:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-12-09T11:38:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-09T11:38:05.390 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.397+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.40 2019-12-09T11:38:05.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.492+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3]) start _multipart_[] num_entries 1001 2019-12-09T11:38:05.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.492+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:05.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.507+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.570+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.570+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-12-09T11:38:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-12-09T11:38:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:05.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:05.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-12-09T11:38:05.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.583+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-12-09T11:38:05.588 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.598+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94152258229504 notifier 4517 bl.length()=198 2019-12-09T11:38:05.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.809+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3 from cache 2019-12-09T11:38:05.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.809+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=255 2019-12-09T11:38:05.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:05.822+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 94152258231056 notifier 4517 bl.length()=255 2019-12-09T11:38:06.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.480+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55a187f1ab80 s->prefetch_data=0 2019-12-09T11:38:06.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.480+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-12-09T11:38:06.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-12-09T11:38:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T11:38:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.493+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:06.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.504+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=231 2019-12-09T11:38:06.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.504+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-12-09T11:38:06.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.504+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T11:38:06.496 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.504+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:06.496 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.504+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 : miss 2019-12-09T11:38:06.511 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 info.flags=0x16 2019-12-09T11:38:06.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 to cache LRU end 2019-12-09T11:38:06.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:06.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:06.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:06.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:06.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:06.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:06.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.520+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:06.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=323 2019-12-09T11:38:06.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 info.flags=0x13 2019-12-09T11:38:06.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 to cache LRU end 2019-12-09T11:38:06.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:06.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:06.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.538+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:06.554 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.563+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4]) start [] num_entries 1100 2019-12-09T11:38:06.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.618+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo[ZSnncz-aUfAsBOFptftuzc4TQC9A0yF] 2019-12-09T11:38:06.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.618+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:06.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.642+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x55a187dc76e8 s->prefetch_data=0 2019-12-09T11:38:06.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.663+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 6b8qs1g6kwrqihrne1o1sxcltu79fb0x 2019-12-09T11:38:06.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.663+0000 7fd8f5d20ec0 20 get_obj_state_impl: setting s->olh_tag to 8pwepw3txfxaxd3u9j7j72t06xfplp37 2019-12-09T11:38:06.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.663+0000 7fd8f5d20ec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=8pwepw3txfxaxd3u9j7j72t06xfplp37 2019-12-09T11:38:06.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.791+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:06.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.865+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:06.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.865+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:06.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.878+0000 7fd8f5d20ec0 20 olh_log_entry: epoch=3 op=2 key=[] 2019-12-09T11:38:06.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.878+0000 7fd8f5d20ec0 20 olh_log_entry: epoch=3 op=3 key=foo[ZSnncz-aUfAsBOFptftuzc4TQC9A0yF] 2019-12-09T11:38:06.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.878+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-index-versioning.1:_:ZSnncz-aUfAsBOFptftuzc4TQC9A0yF_foo state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:06.878 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.887+0000 7fd8f5d20ec0 10 manifest: total_size = 74198 2019-12-09T11:38:06.878 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.887+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.55 2019-12-09T11:38:06.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.887+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-index-versioning.1:_:ZSnncz-aUfAsBOFptftuzc4TQC9A0yF_foo state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:06.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.887+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1be48 2019-12-09T11:38:06.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:06.887+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:07.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-12-09T11:38:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-12-09T11:38:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-12-09T11:38:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-12-09T11:38:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-12-09T11:38:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-12-09T11:38:07.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.015+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.4tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.55 2019-12-09T11:38:07.182 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.191+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:07.183 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.191+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:07.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.283+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:07.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.327+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 2019-12-09T11:38:07.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.327+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4]) start _multipart_[] num_entries 1001 2019-12-09T11:38:07.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.327+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:07.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.340+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:07.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.403+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x55a187dc7050 s->prefetch_data=0 2019-12-09T11:38:07.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.403+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:07.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-12-09T11:38:07.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-12-09T11:38:07.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:07.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:07.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:07.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-12-09T11:38:07.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.415+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-12-09T11:38:07.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:07.433+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 94152258229504 notifier 4517 bl.length()=193 2019-12-09T11:38:08.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.230+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4 from cache 2019-12-09T11:38:08.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.230+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=250 2019-12-09T11:38:08.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.245+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94152258236320 notifier 4517 bl.length()=250 2019-12-09T11:38:08.458 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.467+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:08.458 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.467+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-12-09T11:38:08.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-12-09T11:38:08.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T11:38:08.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:08.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:08.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:08.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.480+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:08.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.496+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T11:38:08.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.496+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-12-09T11:38:08.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.496+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T11:38:08.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.496+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 state=0x55a187f1abc0 s->prefetch_data=0 2019-12-09T11:38:08.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.496+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 : miss 2019-12-09T11:38:08.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 info.flags=0x16 2019-12-09T11:38:08.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 to cache LRU end 2019-12-09T11:38:08.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:08.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:08.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:08.507 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:08.507 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:08.507 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:08.507 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.514+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:08.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.542+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T11:38:08.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.543+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 info.flags=0x13 2019-12-09T11:38:08.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.543+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 to cache LRU end 2019-12-09T11:38:08.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.543+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:08.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.543+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:08.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.543+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 2019-12-09T11:38:08.561 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.570+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5]) start [] num_entries 1100 2019-12-09T11:38:08.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.608+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-12-09T11:38:08.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.608+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.652+0000 7fd8f5d20ec0 10 manifest: total_size = 886297 2019-12-09T11:38:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.652+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.71 2019-12-09T11:38:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.652+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-multipart-simple.1:foo state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:08.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.652+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:08.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.652+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 2019-12-09T11:38:08.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T11:38:08.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=246247 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=246247 stripe_ofs=246247 part_ofs=246247 rule->part_size=246247 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4440551 part_ofs=246247 rule->part_size=246247 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=492494 stripe_ofs=492494 part_ofs=492494 rule->part_size=246247 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=246247 rules.size()=2 2019-12-09T11:38:08.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4686798 part_ofs=492494 rule->part_size=246247 2019-12-09T11:38:08.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=738741 stripe_ofs=738741 part_ofs=738741 rule->part_size=147556 2019-12-09T11:38:08.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=147556 rules.size()=2 2019-12-09T11:38:08.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4933045 part_ofs=738741 rule->part_size=147556 2019-12-09T11:38:08.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=886297 stripe_ofs=886297 part_ofs=886297 rule->part_size=147556 2019-12-09T11:38:08.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.797+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.16tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.71 2019-12-09T11:38:08.874 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.884+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5]) start _multipart_[] num_entries 1001 2019-12-09T11:38:08.874 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.884+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:08.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:08.922+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:08.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.000+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x55a187dc7a00 s->prefetch_data=0 2019-12-09T11:38:08.991 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.000+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:09.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-12-09T11:38:09.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.010+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-12-09T11:38:09.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.029+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584969 cookie 94152258226480 notifier 4517 bl.length()=183 2019-12-09T11:38:09.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.432+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5 from cache 2019-12-09T11:38:09.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.432+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=240 2019-12-09T11:38:09.441 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.451+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94152258236960 notifier 4517 bl.length()=240 2019-12-09T11:38:09.965 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.975+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:09.965 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.975+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-12-09T11:38:09.984 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-12-09T11:38:09.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T11:38:09.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:09.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:09.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:09.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:09.992+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:09.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.009+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=227 2019-12-09T11:38:10.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.009+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-12-09T11:38:10.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.009+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T11:38:10.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.009+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 state=0x55a187f1abc0 s->prefetch_data=0 2019-12-09T11:38:10.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.009+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 : miss 2019-12-09T11:38:10.020 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 info.flags=0x16 2019-12-09T11:38:10.020 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 to cache LRU end 2019-12-09T11:38:10.020 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:10.020 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:10.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:10.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:10.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:10.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:10.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.030+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:10.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.069+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=319 2019-12-09T11:38:10.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.069+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 info.flags=0x13 2019-12-09T11:38:10.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.069+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 to cache LRU end 2019-12-09T11:38:10.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.069+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:10.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.069+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:10.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.070+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:10.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.085+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7]) start [] num_entries 1100 2019-12-09T11:38:10.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T11:38:10.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T11:38:10.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T11:38:10.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T11:38:10.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T11:38:10.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T11:38:10.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T11:38:10.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T11:38:10.103 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T11:38:10.103 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T11:38:10.103 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T11:38:10.103 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T11:38:10.104 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.110+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.117 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.127+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:10.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.127+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.108 2019-12-09T11:38:10.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.127+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.127+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:10.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.127+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:10.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.299+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.299+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.299+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.299+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.299+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.108 2019-12-09T11:38:10.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.356+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:10.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.367+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:10.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.367+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.109 2019-12-09T11:38:10.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.367+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:10.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.367+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1be48 2019-12-09T11:38:10.358 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.367+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:10.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.481+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.481+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.481+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.481+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.481+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.109 2019-12-09T11:38:10.555 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.565+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.576+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:10.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.576+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _U5pfPmLToH1H9IhI9XLLgor8Yr8Hu8M 2019-12-09T11:38:10.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.576+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.576+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:10.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.576+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:10.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.700+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.700+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.700+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.700+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.700+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: _U5pfPmLToH1H9IhI9XLLgor8Yr8Hu8M 2019-12-09T11:38:10.777 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.787+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.790 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.800+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:10.790 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.800+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _0b2r_baSRIzmLiIQiIeJceAAumH-pur 2019-12-09T11:38:10.790 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.800+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:10.791 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.800+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:10.791 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.800+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:10.887 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.897+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:10.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.897+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:10.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:10.897+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.26tag is: _0b2r_baSRIzmLiIQiIeJceAAumH-pur 2019-12-09T11:38:10.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.000+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.009+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:11.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.009+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.112 2019-12-09T11:38:11.000 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.009+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.009+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:11.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.009+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.094+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.112 2019-12-09T11:38:11.133 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.143+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.144 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.154+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:11.144 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.154+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _j1yPZ8nkWMNyN053bUq4J_TNQz_Mfj0 2019-12-09T11:38:11.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.154+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.154+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:11.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.154+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.253+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.253+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.253+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.253+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.253+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: _j1yPZ8nkWMNyN053bUq4J_TNQz_Mfj0 2019-12-09T11:38:11.330 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.340+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.348+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:11.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.348+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.114 2019-12-09T11:38:11.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.348+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.348+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:11.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.348+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.438+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.438+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.438+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.438+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.438+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.114 2019-12-09T11:38:11.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.503+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:11.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.514+0000 7fd8f5d20ec0 10 manifest: total_size = 5242880 2019-12-09T11:38:11.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.514+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _KIvkhsqD44Xg278QRk3YBykaJduTt7a 2019-12-09T11:38:11.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.514+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:11.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.514+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1be48 2019-12-09T11:38:11.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.514+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.662+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.662+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.662+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.1tag is: _KIvkhsqD44Xg278QRk3YBykaJduTt7a 2019-12-09T11:38:11.770 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.780+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.792+0000 7fd8f5d20ec0 10 manifest: total_size = 27195 2019-12-09T11:38:11.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.792+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.116 2019-12-09T11:38:11.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.792+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:11.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.792+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:11.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.792+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.872 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.882+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:11.880 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.890+0000 7fd8f5d20ec0 10 manifest: total_size = 27195 2019-12-09T11:38:11.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.890+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _-gTh5aNFxZnRNx1cE_abbxQz7vSxeFm 2019-12-09T11:38:11.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.890+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x55a187f1be48 s->prefetch_data=0 2019-12-09T11:38:11.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.890+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f1be48 2019-12-09T11:38:11.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:11.890+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:11.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.004+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:11.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.004+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=27195 stripe_ofs=27195 part_ofs=0 rule->part_size=0 2019-12-09T11:38:11.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.004+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _-gTh5aNFxZnRNx1cE_abbxQz7vSxeFm 2019-12-09T11:38:12.048 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.058+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:12.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.069+0000 7fd8f5d20ec0 10 manifest: total_size = 27195 2019-12-09T11:38:12.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.069+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _aDfXTllXMftoD6j3Qq9ZQUF1RP9fZQL 2019-12-09T11:38:12.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.069+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:12.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.069+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:12.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.069+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:12.166 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.176+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:12.166 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.176+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=27195 stripe_ofs=27195 part_ofs=0 rule->part_size=0 2019-12-09T11:38:12.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.176+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _aDfXTllXMftoD6j3Qq9ZQUF1RP9fZQL 2019-12-09T11:38:12.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.226+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:12.228 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.235+0000 7fd8f5d20ec0 10 manifest: total_size = 27195 2019-12-09T11:38:12.228 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.235+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _WiY3Als2sk1p96mxPlKDrbtxPBm57py 2019-12-09T11:38:12.229 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.235+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:12.229 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.235+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:12.229 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.235+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 2019-12-09T11:38:12.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.319+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7]) start _multipart_[] num_entries 1001 2019-12-09T11:38:12.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.319+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:12.366 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.376+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:12.411 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.421+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x55a187f18e00 s->prefetch_data=0 2019-12-09T11:38:12.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.421+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:12.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-12-09T11:38:12.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-12-09T11:38:12.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:12.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:12.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:12.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-12-09T11:38:12.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.433+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-12-09T11:38:12.433 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.443+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584970 cookie 94152258229504 notifier 4517 bl.length()=189 2019-12-09T11:38:12.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.835+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7 from cache 2019-12-09T11:38:12.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.835+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=246 2019-12-09T11:38:12.836 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:12.846+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94152258236320 notifier 4517 bl.length()=246 2019-12-09T11:38:13.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.553+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:13.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.553+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-12-09T11:38:13.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-12-09T11:38:13.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T11:38:13.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:13.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:13.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:13.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.566+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:13.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.578+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=237 2019-12-09T11:38:13.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.578+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-12-09T11:38:13.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.578+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T11:38:13.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.578+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 state=0x55a187f1abc0 s->prefetch_data=0 2019-12-09T11:38:13.570 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.578+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 : miss 2019-12-09T11:38:13.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 info.flags=0x16 2019-12-09T11:38:13.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 to cache LRU end 2019-12-09T11:38:13.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:13.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:13.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:13.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:13.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:13.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:13.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.592+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=329 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 info.flags=0x13 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 to cache LRU end 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:13.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.610+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:13.624 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.634+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9]) start [] num_entries 1100 2019-12-09T11:38:13.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T11:38:13.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-12-09T11:38:13.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-12-09T11:38:13.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-12-09T11:38:13.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T11:38:13.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-12-09T11:38:13.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-12-09T11:38:13.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-12-09T11:38:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T11:38:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-12-09T11:38:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-12-09T11:38:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-12-09T11:38:13.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.659+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.659 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.669+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:13.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.669+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.224 2019-12-09T11:38:13.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.669+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.669+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:13.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.669+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:13.747 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.747 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:13.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:13.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:13.751 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:13.751 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.755+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.224 2019-12-09T11:38:13.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.822+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.831+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:13.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.831+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.225 2019-12-09T11:38:13.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.831+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.831+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:13.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.831+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:13.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:13.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:13.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:13.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.919+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.7tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.225 2019-12-09T11:38:13.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.985+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.995+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:13.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.995+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to __aYit9ZDRdeeE85eAjbezHXKSCnz8Ki 2019-12-09T11:38:13.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.995+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:13.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.995+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:13.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:13.995+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:14.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.094+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: __aYit9ZDRdeeE85eAjbezHXKSCnz8Ki 2019-12-09T11:38:14.182 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.192+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.191 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.201+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:14.191 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.201+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _ogy-Je_f93SfK-4gI6H1CcBVnWZdJoH 2019-12-09T11:38:14.191 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.201+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.191 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.201+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:14.192 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.201+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:14.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.426+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.426+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.426+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.426+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.426+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: _ogy-Je_f93SfK-4gI6H1CcBVnWZdJoH 2019-12-09T11:38:14.503 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.512+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.522+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:14.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.522+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.233 2019-12-09T11:38:14.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.522+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.522+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:14.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.522+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:14.607 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:14.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:14.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:14.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:14.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:14.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:14.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:14.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:14.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:14.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.617+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.233 2019-12-09T11:38:14.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.687+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.699+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.699+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _1Q4UA6FhCSVLNqMp4rXpKtLhMYFHPby 2019-12-09T11:38:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.699+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.699+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.699+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:14.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.830 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:14.830 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T11:38:14.830 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.838+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: _1Q4UA6FhCSVLNqMp4rXpKtLhMYFHPby 2019-12-09T11:38:14.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.945+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.957+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:14.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.957+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.235 2019-12-09T11:38:14.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.957+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:14.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.957+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:14.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:14.957+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:15.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:15.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:15.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:15.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:15.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:15.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:15.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:15.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:15.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:15.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.071+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.235 2019-12-09T11:38:15.135 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.145+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.158+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:15.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.158+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _CwCU1s1pGcbJMWNZG7VZmw8JSB5fR3P 2019-12-09T11:38:15.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.158+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.158+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:15.150 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.158+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.288+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.288+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.288+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.288+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-12-09T11:38:15.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.288+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.11tag is: _CwCU1s1pGcbJMWNZG7VZmw8JSB5fR3P 2019-12-09T11:38:15.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.380+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.408+0000 7fd8f5d20ec0 10 manifest: total_size = 16379 2019-12-09T11:38:15.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.408+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.242 2019-12-09T11:38:15.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.408+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.408+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:15.400 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.408+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:15.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=2 2019-12-09T11:38:15.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7672 2019-12-09T11:38:15.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7672 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=2 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4201976 part_ofs=7672 rule->part_size=7672 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=15344 stripe_ofs=15344 part_ofs=15344 rule->part_size=1035 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=2 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4209648 part_ofs=15344 rule->part_size=1035 2019-12-09T11:38:15.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=16379 stripe_ofs=16379 part_ofs=16379 rule->part_size=1035 2019-12-09T11:38:15.500 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.507+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.3tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.242 2019-12-09T11:38:15.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.599+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.608+0000 7fd8f5d20ec0 10 manifest: total_size = 16379 2019-12-09T11:38:15.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.608+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _DUai0F0zcEH7aFWd4xPvrxDw6bgBKiF 2019-12-09T11:38:15.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.608+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.608+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:15.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.608+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:15.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.692+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:15.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.692+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=16379 stripe_ofs=16379 part_ofs=0 rule->part_size=0 2019-12-09T11:38:15.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.692+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.10tag is: _DUai0F0zcEH7aFWd4xPvrxDw6bgBKiF 2019-12-09T11:38:15.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.767+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.783+0000 7fd8f5d20ec0 10 manifest: total_size = 16379 2019-12-09T11:38:15.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.783+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to _loTPLdPFdXl7iD3eWBCQTbyl_SNfNNp 2019-12-09T11:38:15.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.783+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.783+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:15.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.783+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.891+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.891+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=16379 stripe_ofs=16379 part_ofs=0 rule->part_size=0 2019-12-09T11:38:15.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.891+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: _loTPLdPFdXl7iD3eWBCQTbyl_SNfNNp 2019-12-09T11:38:15.961 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.970+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.985+0000 7fd8f5d20ec0 10 manifest: total_size = 16379 2019-12-09T11:38:15.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.985+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.245 2019-12-09T11:38:15.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.985+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x55a187f180c8 s->prefetch_data=0 2019-12-09T11:38:15.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.985+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f180c8 2019-12-09T11:38:15.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:15.985+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 2019-12-09T11:38:16.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=2 2019-12-09T11:38:16.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7672 2019-12-09T11:38:16.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=7672 stripe_ofs=7672 part_ofs=7672 rule->part_size=7672 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7672 rules.size()=2 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4201976 part_ofs=7672 rule->part_size=7672 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=15344 stripe_ofs=15344 part_ofs=15344 rule->part_size=1035 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=2 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4209648 part_ofs=15344 rule->part_size=1035 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=16379 stripe_ofs=16379 part_ofs=16379 rule->part_size=1035 2019-12-09T11:38:16.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.123+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.245 2019-12-09T11:38:16.227 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.236+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9]) start _multipart_[] num_entries 1001 2019-12-09T11:38:16.227 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.237+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:16.252 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.262+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:16.328 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.338+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x55a187dc7690 s->prefetch_data=0 2019-12-09T11:38:16.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.338+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:16.342 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-12-09T11:38:16.342 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-12-09T11:38:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-12-09T11:38:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.350+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-12-09T11:38:16.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.358+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 94152258231056 notifier 4517 bl.length()=199 2019-12-09T11:38:16.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.833+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9 from cache 2019-12-09T11:38:16.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.833+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=256 2019-12-09T11:38:16.867 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:16.877+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 94152258239920 notifier 4517 bl.length()=256 2019-12-09T11:38:17.664 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.674+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:17.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.674+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-12-09T11:38:17.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-12-09T11:38:17.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T11:38:17.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:17.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:17.675 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:17.675 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.683+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:17.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.694+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=232 2019-12-09T11:38:17.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.694+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-12-09T11:38:17.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.694+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T11:38:17.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.694+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 state=0x55a187f1a890 s->prefetch_data=0 2019-12-09T11:38:17.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.694+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 : miss 2019-12-09T11:38:17.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 info.flags=0x16 2019-12-09T11:38:17.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 to cache LRU end 2019-12-09T11:38:17.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:17.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:17.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:17.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:17.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:17.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:17.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.712+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:17.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=324 2019-12-09T11:38:17.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 info.flags=0x13 2019-12-09T11:38:17.722 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 to cache LRU end 2019-12-09T11:38:17.722 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:17.722 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:17.722 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.730+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 2019-12-09T11:38:17.734 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.744+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8]) start [] num_entries 1100 2019-12-09T11:38:17.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.761+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T11:38:17.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.761+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T11:38:17.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.761+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T11:38:17.753 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.761+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:17.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.773+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:17.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.773+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.196 2019-12-09T11:38:17.764 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.773+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:17.764 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.773+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18438 2019-12-09T11:38:17.764 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.773+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 2019-12-09T11:38:17.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:17.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:17.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:17.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:17.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:17.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:17.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:17.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:17.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:17.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.900+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.196 2019-12-09T11:38:17.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.982+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:17.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.995+0000 7fd8f5d20ec0 10 manifest: total_size = 11534336 2019-12-09T11:38:17.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.995+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.203 2019-12-09T11:38:17.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.995+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:17.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.995+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18438 2019-12-09T11:38:17.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:17.995+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 2019-12-09T11:38:18.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:18.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:18.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:18.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:18.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-12-09T11:38:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-12-09T11:38:18.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:18.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-12-09T11:38:18.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-12-09T11:38:18.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-12-09T11:38:18.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.121+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.5tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.203 2019-12-09T11:38:18.211 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.220+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:18.225 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.235+0000 7fd8f5d20ec0 10 manifest: total_size = 16376 2019-12-09T11:38:18.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.235+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.210 2019-12-09T11:38:18.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.235+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x55a187f18438 s->prefetch_data=0 2019-12-09T11:38:18.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.235+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18438 2019-12-09T11:38:18.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.235+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 2019-12-09T11:38:18.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7671 rules.size()=3 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=7671 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=7671 stripe_ofs=7671 part_ofs=7671 rule->part_size=7670 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=7670 rules.size()=3 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4201975 part_ofs=7671 rule->part_size=7670 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=15341 stripe_ofs=15341 part_ofs=15341 rule->part_size=1035 2019-12-09T11:38:18.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=1035 rules.size()=3 2019-12-09T11:38:18.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): stripe_ofs=4209645 part_ofs=15341 rule->part_size=1035 2019-12-09T11:38:18.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=16376 stripe_ofs=16376 part_ofs=16376 rule->part_size=1035 2019-12-09T11:38:18.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.349+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.27tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.210 2019-12-09T11:38:18.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.423+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8]) start _multipart_[] num_entries 1001 2019-12-09T11:38:18.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.423+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:18.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.432+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:18.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.497+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:18.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.497+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:18.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-12-09T11:38:18.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-12-09T11:38:18.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:18.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:18.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:18.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-12-09T11:38:18.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.508+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-12-09T11:38:18.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.524+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94152258236320 notifier 4517 bl.length()=194 2019-12-09T11:38:18.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.713+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8 from cache 2019-12-09T11:38:18.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.713+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=251 2019-12-09T11:38:18.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:18.726+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94152258239920 notifier 4517 bl.length()=251 2019-12-09T11:38:19.120 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.130+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:19.120 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.130+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-12-09T11:38:19.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-12-09T11:38:19.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T11:38:19.141 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:19.141 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:19.141 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:19.141 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.150+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:19.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.162+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=222 2019-12-09T11:38:19.153 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.162+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-12-09T11:38:19.153 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.162+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T11:38:19.153 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.162+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 state=0x55a187dc7690 s->prefetch_data=0 2019-12-09T11:38:19.153 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.162+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 : miss 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 info.flags=0x16 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 to cache LRU end 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:19.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:19.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:19.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:19.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.183+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:19.203 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=314 2019-12-09T11:38:19.203 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 info.flags=0x13 2019-12-09T11:38:19.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 to cache LRU end 2019-12-09T11:38:19.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:19.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:19.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.213+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 2019-12-09T11:38:19.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.225+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6]) start [] num_entries 1100 2019-12-09T11:38:19.241 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.251+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-12-09T11:38:19.241 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.251+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-12-09T11:38:19.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.251+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-12-09T11:38:19.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.251+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.265+0000 7fd8f5d20ec0 10 manifest: total_size = 10485760 2019-12-09T11:38:19.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.265+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.84 2019-12-09T11:38:19.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.265+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.265+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18808 2019-12-09T11:38:19.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.265+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 2019-12-09T11:38:19.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.363+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.84 2019-12-09T11:38:19.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.446+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.444 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.453+0000 7fd8f5d20ec0 10 manifest: total_size = 10485760 2019-12-09T11:38:19.444 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.453+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.85 2019-12-09T11:38:19.444 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.453+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.444 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.453+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18808 2019-12-09T11:38:19.444 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.453+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 2019-12-09T11:38:19.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-12-09T11:38:19.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-12-09T11:38:19.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.565+0000 7fd8f5d20ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.12tag is: 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.85 2019-12-09T11:38:19.634 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.644+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.652+0000 7fd8f5d20ec0 10 manifest: total_size = 648332 2019-12-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.652+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.86 2019-12-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.652+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x55a187f18808 s->prefetch_data=0 2019-12-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.652+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18808 2019-12-09T11:38:19.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.652+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 2019-12-09T11:38:19.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.759+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6]) start _multipart_[] num_entries 1001 2019-12-09T11:38:19.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.759+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:19.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.843+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:19.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.902+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:19.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.902+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:19.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-12-09T11:38:19.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.938+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-12-09T11:38:19.940 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:19.949+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584971 cookie 94152258229504 notifier 4517 bl.length()=184 2019-12-09T11:38:20.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.368+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6 from cache 2019-12-09T11:38:20.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.368+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=241 2019-12-09T11:38:20.371 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.379+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94152258231056 notifier 4517 bl.length()=241 2019-12-09T11:38:20.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.565+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:20.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.565+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:20.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.580+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:20.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.595+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T11:38:20.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.595+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-12-09T11:38:20.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.595+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T11:38:20.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.595+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 state=0x55a187dc7050 s->prefetch_data=0 2019-12-09T11:38:20.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.595+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 : miss 2019-12-09T11:38:20.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 info.flags=0x16 2019-12-09T11:38:20.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 to cache LRU end 2019-12-09T11:38:20.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:20.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:20.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:20.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:20.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:20.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:20.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.608+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:20.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T11:38:20.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 info.flags=0x13 2019-12-09T11:38:20.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 to cache LRU end 2019-12-09T11:38:20.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:20.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:20.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.634+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 2019-12-09T11:38:20.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.658+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10]) start [] num_entries 1100 2019-12-09T11:38:20.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.701+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T11:38:20.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.701+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T11:38:20.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.701+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T11:38:20.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.701+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T11:38:20.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.701+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:20.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.723+0000 7fd8f5d20ec0 10 manifest: total_size = 1035 2019-12-09T11:38:20.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.723+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.305 2019-12-09T11:38:20.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.723+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:__ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:20.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.723+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:20.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.723+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 2019-12-09T11:38:20.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.920+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.935+0000 7fd8f5d20ec0 10 manifest: total_size = 110 2019-12-09T11:38:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.935+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.307 2019-12-09T11:38:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.935+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.935+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:20.935+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 2019-12-09T11:38:21.105 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.113+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:21.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.127+0000 7fd8f5d20ec0 10 manifest: total_size = 110 2019-12-09T11:38:21.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.127+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.306 2019-12-09T11:38:21.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.127+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:___ state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:21.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.127+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:21.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.127+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 2019-12-09T11:38:21.248 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.256+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:21.264 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.273+0000 7fd8f5d20ec0 10 manifest: total_size = 1048576 2019-12-09T11:38:21.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.273+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.304 2019-12-09T11:38:21.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.273+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.1:obj state=0x55a187f19478 s->prefetch_data=0 2019-12-09T11:38:21.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.273+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f19478 2019-12-09T11:38:21.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.273+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 2019-12-09T11:38:21.389 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.397+0000 7fd8b37de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:38:21.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.421+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10]) start _multipart_[] num_entries 1001 2019-12-09T11:38:21.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.421+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:21.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.502+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:21.540 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.548+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x55a187f1bf90 s->prefetch_data=0 2019-12-09T11:38:21.540 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.548+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:21.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-12-09T11:38:21.551 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.558+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T11:38:21.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:21.565+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94152258224992 notifier 4517 bl.length()=181 2019-12-09T11:38:22.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.085+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.10 from cache 2019-12-09T11:38:22.077 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.085+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-12-09T11:38:22.089 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.097+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 94152258236320 notifier 4517 bl.length()=239 2019-12-09T11:38:22.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.610+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55a187dc7050 s->prefetch_data=0 2019-12-09T11:38:22.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.610+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:22.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.625+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:22.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.661+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T11:38:22.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.661+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-12-09T11:38:22.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.661+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T11:38:22.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.661+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:22.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.661+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 : miss 2019-12-09T11:38:22.667 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 info.flags=0x16 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 to cache LRU end 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:22.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:22.669 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:22.669 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.674+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:22.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T11:38:22.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 info.flags=0x13 2019-12-09T11:38:22.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 to cache LRU end 2019-12-09T11:38:22.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:22.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:22.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.691+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 2019-12-09T11:38:22.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.714+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11]) start [] num_entries 1100 2019-12-09T11:38:22.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.754+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T11:38:22.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.754+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T11:38:22.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.754+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T11:38:22.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.754+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T11:38:22.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.754+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:22.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.775+0000 7fd8f5d20ec0 10 manifest: total_size = 62 2019-12-09T11:38:22.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.775+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.312 2019-12-09T11:38:22.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.775+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:__ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:22.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.775+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc6058 2019-12-09T11:38:22.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.775+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 2019-12-09T11:38:22.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.926+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:22.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.941+0000 7fd8f5d20ec0 10 manifest: total_size = 527 2019-12-09T11:38:22.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.941+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.314 2019-12-09T11:38:22.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.941+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:22.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.941+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc6058 2019-12-09T11:38:22.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:22.941+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 2019-12-09T11:38:23.049 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.057+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:23.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.074+0000 7fd8f5d20ec0 10 manifest: total_size = 2078 2019-12-09T11:38:23.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.074+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.313 2019-12-09T11:38:23.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.074+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:___ state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:23.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.074+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc6058 2019-12-09T11:38:23.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.074+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 2019-12-09T11:38:23.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.244+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:23.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.257+0000 7fd8f5d20ec0 10 manifest: total_size = 2078 2019-12-09T11:38:23.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.257+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.311 2019-12-09T11:38:23.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.257+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.2:obj state=0x55a187dc6058 s->prefetch_data=0 2019-12-09T11:38:23.250 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.257+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187dc6058 2019-12-09T11:38:23.250 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.257+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 2019-12-09T11:38:23.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.500+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11]) start _multipart_[] num_entries 1001 2019-12-09T11:38:23.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.500+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:23.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.595+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:23.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.664+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x55a187dc7050 s->prefetch_data=0 2019-12-09T11:38:23.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.664+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:23.670 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-12-09T11:38:23.670 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-12-09T11:38:23.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:23.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:23.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:23.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-12-09T11:38:23.672 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.678+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T11:38:23.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.691+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584969 cookie 94152258224992 notifier 4517 bl.length()=181 2019-12-09T11:38:23.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.889+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.11 from cache 2019-12-09T11:38:23.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.890+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-12-09T11:38:23.895 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:23.903+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584970 cookie 94152258236320 notifier 4517 bl.length()=239 2019-12-09T11:38:24.516 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.524+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55a187dc7050 s->prefetch_data=0 2019-12-09T11:38:24.516 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.524+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-12-09T11:38:24.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-12-09T11:38:24.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T11:38:24.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:24.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:24.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-12-09T11:38:24.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.538+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:24.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.554+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=221 2019-12-09T11:38:24.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.554+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-12-09T11:38:24.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.554+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T11:38:24.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.554+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b90450 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:24.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.554+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 : miss 2019-12-09T11:38:24.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 info.flags=0x16 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 to cache LRU end 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.acl 2019-12-09T11:38:24.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 20 Read xattr: user.rgw.idtag 2019-12-09T11:38:24.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 : type miss (requested=0x13, cached=0x16) 2019-12-09T11:38:24.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.575+0000 7fd8f5d20ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:24.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 20 rados_obj.operate() r=0 bl.length=313 2019-12-09T11:38:24.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 info.flags=0x13 2019-12-09T11:38:24.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 to cache LRU end 2019-12-09T11:38:24.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:24.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-12-09T11:38:24.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.598+0000 7fd8f5d20ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 2019-12-09T11:38:24.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.625+0000 7fd8f5d20ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12]) start [] num_entries 1100 2019-12-09T11:38:24.632 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.641+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-12-09T11:38:24.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.641+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-12-09T11:38:24.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.641+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-12-09T11:38:24.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.641+0000 7fd8f5d20ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-12-09T11:38:24.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.641+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:24.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.659+0000 7fd8f5d20ec0 10 manifest: total_size = 0 2019-12-09T11:38:24.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.659+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.319 2019-12-09T11:38:24.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.659+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:__ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:24.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.659+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18418 2019-12-09T11:38:24.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.659+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 2019-12-09T11:38:24.795 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.803+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.840+0000 7fd8f5d20ec0 10 manifest: total_size = 0 2019-12-09T11:38:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.840+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.321 2019-12-09T11:38:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.840+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.840+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18418 2019-12-09T11:38:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:24.840+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 2019-12-09T11:38:24.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.002+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:25.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.014+0000 7fd8f5d20ec0 10 manifest: total_size = 0 2019-12-09T11:38:25.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.014+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.320 2019-12-09T11:38:25.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.014+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:___ state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:25.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.014+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18418 2019-12-09T11:38:25.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.014+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 2019-12-09T11:38:25.131 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.140+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:25.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.153+0000 7fd8f5d20ec0 10 manifest: total_size = 0 2019-12-09T11:38:25.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.153+0000 7fd8f5d20ec0 20 get_obj_state: setting s->obj_tag to 7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.318 2019-12-09T11:38:25.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.153+0000 7fd8f5d20ec0 20 get_obj_state: rctx=0x7ffc19b90470 obj=test-client.0-r-test-small-obj-data.3:obj state=0x55a187f18418 s->prefetch_data=0 2019-12-09T11:38:25.146 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.153+0000 7fd8f5d20ec0 20 prepare_atomic_modification: state is not atomic. state=0x55a187f18418 2019-12-09T11:38:25.146 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.154+0000 7fd8f5d20ec0 20 bucket index object: default.rgw.buckets.index:.dir.7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 2019-12-09T11:38:25.350 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.358+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12]) start _multipart_[] num_entries 1001 2019-12-09T11:38:25.376 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.358+0000 7fd8f5d20ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-12-09T11:38:25.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.426+0000 7fd8f5d20ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12])"; objs.size()=0; is_truncated=0 2019-12-09T11:38:25.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.509+0000 7fd8f5d20ec0 20 get_system_obj_state: rctx=0x7ffc19b8fe08 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x55a187f1cc90 s->prefetch_data=0 2019-12-09T11:38:25.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.509+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-12-09T11:38:25.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-12-09T11:38:25.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.522+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-12-09T11:38:25.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.535+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584971 cookie 94152258224992 notifier 4517 bl.length()=181 2019-12-09T11:38:25.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.947+0000 7fd8f5d20ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.12 from cache 2019-12-09T11:38:25.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.947+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-12-09T11:38:25.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:25.961+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584972 cookie 94152258236320 notifier 4517 bl.length()=239 2019-12-09T11:38:26.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:26.968+0000 7fd8f5d20ec0 10 removing key index: NZYBRWCJCEYDZQXBSMTK 2019-12-09T11:38:26.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:26.968+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-12-09T11:38:26.987 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:26.996+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584973 cookie 94152258236320 notifier 4517 bl.length()=170 2019-12-09T11:38:27.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.408+0000 7fd8f5d20ec0 10 removing email index: ragweed.client.0+test@test.test 2019-12-09T11:38:27.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.408+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-12-09T11:38:27.408 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.418+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584972 cookie 94152258229504 notifier 4517 bl.length()=182 2019-12-09T11:38:27.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.967+0000 7fd8f5d20ec0 10 removing user buckets index 2019-12-09T11:38:27.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.967+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-12-09T11:38:27.966 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:27.976+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584974 cookie 94152258236320 notifier 4517 bl.length()=173 2019-12-09T11:38:28.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.372+0000 7fd8f5d20ec0 10 removing user index: ragweed.client.0 2019-12-09T11:38:28.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.372+0000 7fd8f5d20ec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-12-09T11:38:28.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.372+0000 7fd8f5d20ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-12-09T11:38:28.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.388+0000 7fd8c3fff700 10 RGWWatcher::handle_notify() notify_id 579820584973 cookie 94152258226480 notifier 4517 bl.length()=165 2019-12-09T11:38:28.720 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.729+0000 7fd8f5d20ec0 20 remove_watcher() i=0 2019-12-09T11:38:28.720 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.729+0000 7fd8f5d20ec0 2 removed watcher, disabling cache 2019-12-09T11:38:28.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.791+0000 7fd8f5d20ec0 20 remove_watcher() i=1 2019-12-09T11:38:28.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.848+0000 7fd8f5d20ec0 20 remove_watcher() i=2 2019-12-09T11:38:28.885 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.894+0000 7fd8f5d20ec0 20 remove_watcher() i=3 2019-12-09T11:38:28.946 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.955+0000 7fd8f5d20ec0 20 remove_watcher() i=4 2019-12-09T11:38:28.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:28.994+0000 7fd8f5d20ec0 20 remove_watcher() i=5 2019-12-09T11:38:29.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:29.055+0000 7fd8f5d20ec0 20 remove_watcher() i=6 2019-12-09T11:38:29.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:29.095+0000 7fd8f5d20ec0 20 remove_watcher() i=7 2019-12-09T11:38:29.092 INFO:tasks.ragweed:Removing ragweed... 2019-12-09T11:38:29.092 INFO:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/ragweed 2019-12-09T11:38:29.187 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-12-09T11:38:29.209 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-12-09T11:38:29.210 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-12-09T11:38:29.210 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-12-09T11:38:29.210 INFO:teuthology.orchestra.run.smithi174:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-12-09T11:38:29.258 INFO:teuthology.orchestra.run.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-12-09T11:38:32.333 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:17:24.181 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.375 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:24.211 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.376 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:24.212 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.379 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:24.216 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.382 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:17:24.230 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.407 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:24.244 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.422 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:17:24.258 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.450 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:17:24.286 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:32.461 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:17:24.309 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:38:33.010 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-12-09T11:38:33.010 INFO:teuthology.orchestra.run.smithi174:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-12-09T11:38:35.315 INFO:teuthology.orchestra.run.smithi174.stdout: Gradle will be installed under /opt/gradle 2019-12-09T11:38:35.372 INFO:teuthology.orchestra.run.smithi174.stderr:--2019-12-09 11:38:35-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-12-09T11:38:35.381 INFO:teuthology.orchestra.run.smithi174.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.191.9, 104.18.190.9, 2606:4700::6812:be09, ... 2019-12-09T11:38:35.389 INFO:teuthology.orchestra.run.smithi174.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.191.9|:443... connected. 2019-12-09T11:38:35.433 INFO:teuthology.orchestra.run.smithi174.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-12-09T11:38:35.434 INFO:teuthology.orchestra.run.smithi174.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-12-09T11:38:35.434 INFO:teuthology.orchestra.run.smithi174.stderr:--2019-12-09 11:38:35-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-12-09T11:38:35.434 INFO:teuthology.orchestra.run.smithi174.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.159.20, 104.17.160.20, 2606:4700::6811:9f14, ... 2019-12-09T11:38:35.452 INFO:teuthology.orchestra.run.smithi174.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.159.20|:443... connected. 2019-12-09T11:38:35.488 INFO:teuthology.orchestra.run.smithi174.stderr:HTTP request sent, awaiting response... 200 OK 2019-12-09T11:38:35.488 INFO:teuthology.orchestra.run.smithi174.stderr:Length: 75323957 (72M) [application/zip] 2019-12-09T11:38:35.488 INFO:teuthology.orchestra.run.smithi174.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-12-09T11:38:35.488 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T11:38:35.495 INFO:teuthology.orchestra.run.smithi174.stderr: 0K .......... .......... .......... .......... .......... 0% 6.59M 11s 2019-12-09T11:38:35.496 INFO:teuthology.orchestra.run.smithi174.stderr: 50K .......... .......... .......... .......... .......... 0% 161M 6s 2019-12-09T11:38:35.502 INFO:teuthology.orchestra.run.smithi174.stderr: 100K .......... .......... .......... .......... .......... 0% 6.86M 7s 2019-12-09T11:38:35.502 INFO:teuthology.orchestra.run.smithi174.stderr: 150K .......... .......... .......... .......... .......... 0% 146M 6s 2019-12-09T11:38:35.505 INFO:teuthology.orchestra.run.smithi174.stderr: 200K .......... .......... .......... .......... .......... 0% 144M 5s 2019-12-09T11:38:35.505 INFO:teuthology.orchestra.run.smithi174.stderr: 250K .......... .......... .......... .......... .......... 0% 126M 4s 2019-12-09T11:38:35.505 INFO:teuthology.orchestra.run.smithi174.stderr: 300K .......... .......... .......... .......... .......... 0% 141M 3s 2019-12-09T11:38:35.510 INFO:teuthology.orchestra.run.smithi174.stderr: 350K .......... .......... .......... .......... .......... 0% 7.69M 4s 2019-12-09T11:38:35.510 INFO:teuthology.orchestra.run.smithi174.stderr: 400K .......... .......... .......... .......... .......... 0% 150M 4s 2019-12-09T11:38:35.515 INFO:teuthology.orchestra.run.smithi174.stderr: 450K .......... .......... .......... .......... .......... 0% 115M 3s 2019-12-09T11:38:35.515 INFO:teuthology.orchestra.run.smithi174.stderr: 500K .......... .......... .......... .......... .......... 0% 151M 3s 2019-12-09T11:38:35.515 INFO:teuthology.orchestra.run.smithi174.stderr: 550K .......... .......... .......... .......... .......... 0% 135M 3s 2019-12-09T11:38:35.515 INFO:teuthology.orchestra.run.smithi174.stderr: 600K .......... .......... .......... .......... .......... 0% 153M 3s 2019-12-09T11:38:35.516 INFO:teuthology.orchestra.run.smithi174.stderr: 650K .......... .......... .......... .......... .......... 0% 122M 3s 2019-12-09T11:38:35.516 INFO:teuthology.orchestra.run.smithi174.stderr: 700K .......... .......... .......... .......... .......... 1% 115M 2s 2019-12-09T11:38:35.516 INFO:teuthology.orchestra.run.smithi174.stderr: 750K .......... .......... .......... .......... .......... 1% 111M 2s 2019-12-09T11:38:35.521 INFO:teuthology.orchestra.run.smithi174.stderr: 800K .......... .......... .......... .......... .......... 1% 11.3M 3s 2019-12-09T11:38:35.521 INFO:teuthology.orchestra.run.smithi174.stderr: 850K .......... .......... .......... .......... .......... 1% 100M 2s 2019-12-09T11:38:35.521 INFO:teuthology.orchestra.run.smithi174.stderr: 900K .......... .......... .......... .......... .......... 1% 94.0M 2s 2019-12-09T11:38:35.522 INFO:teuthology.orchestra.run.smithi174.stderr: 950K .......... .......... .......... .......... .......... 1% 187M 2s 2019-12-09T11:38:35.522 INFO:teuthology.orchestra.run.smithi174.stderr: 1000K .......... .......... .......... .......... .......... 1% 154M 2s 2019-12-09T11:38:35.522 INFO:teuthology.orchestra.run.smithi174.stderr: 1050K .......... .......... .......... .......... .......... 1% 124M 2s 2019-12-09T11:38:35.522 INFO:teuthology.orchestra.run.smithi174.stderr: 1100K .......... .......... .......... .......... .......... 1% 110M 2s 2019-12-09T11:38:35.522 INFO:teuthology.orchestra.run.smithi174.stderr: 1150K .......... .......... .......... .......... .......... 1% 110M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1200K .......... .......... .......... .......... .......... 1% 138M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1250K .......... .......... .......... .......... .......... 1% 280M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1300K .......... .......... .......... .......... .......... 1% 140M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1350K .......... .......... .......... .......... .......... 1% 121M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1400K .......... .......... .......... .......... .......... 1% 149M 2s 2019-12-09T11:38:35.532 INFO:teuthology.orchestra.run.smithi174.stderr: 1450K .......... .......... .......... .......... .......... 2% 148M 2s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1500K .......... .......... .......... .......... .......... 2% 148M 2s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1550K .......... .......... .......... .......... .......... 2% 117M 2s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1600K .......... .......... .......... .......... .......... 2% 121M 2s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1650K .......... .......... .......... .......... .......... 2% 141M 2s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1700K .......... .......... .......... .......... .......... 2% 156M 1s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1750K .......... .......... .......... .......... .......... 2% 126M 1s 2019-12-09T11:38:35.533 INFO:teuthology.orchestra.run.smithi174.stderr: 1800K .......... .......... .......... .......... .......... 2% 174M 1s 2019-12-09T11:38:36.999 INFO:teuthology.orchestra.run.smithi174.stderr: 1850K .......... .......... .......... .......... .......... 2% 139M 1s 2019-12-09T11:38:36.999 INFO:teuthology.orchestra.run.smithi174.stderr: 1900K .......... .......... .......... .......... .......... 2% 149M 1s 2019-12-09T11:38:36.999 INFO:teuthology.orchestra.run.smithi174.stderr: 1950K .......... .......... .......... .......... .......... 2% 153M 1s 2019-12-09T11:38:36.999 INFO:teuthology.orchestra.run.smithi174.stderr: 2000K .......... .......... .......... .......... .......... 2% 142M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2050K .......... .......... .......... .......... .......... 2% 149M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2100K .......... .......... .......... .......... .......... 2% 101M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2150K .......... .......... .......... .......... .......... 2% 84.9M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2200K .......... .......... .......... .......... .......... 3% 111M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2250K .......... .......... .......... .......... .......... 3% 119M 1s 2019-12-09T11:38:37.000 INFO:teuthology.orchestra.run.smithi174.stderr: 2300K .......... .......... .......... .......... .......... 3% 125M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2350K .......... .......... .......... .......... .......... 3% 137M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2400K .......... .......... .......... .......... .......... 3% 165M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2450K .......... .......... .......... .......... .......... 3% 134M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2500K .......... .......... .......... .......... .......... 3% 117M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2550K .......... .......... .......... .......... .......... 3% 92.1M 1s 2019-12-09T11:38:37.001 INFO:teuthology.orchestra.run.smithi174.stderr: 2600K .......... .......... .......... .......... .......... 3% 106M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2650K .......... .......... .......... .......... .......... 3% 111M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2700K .......... .......... .......... .......... .......... 3% 114M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2750K .......... .......... .......... .......... .......... 3% 103M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2800K .......... .......... .......... .......... .......... 3% 121M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2850K .......... .......... .......... .......... .......... 3% 113M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2900K .......... .......... .......... .......... .......... 4% 185M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 2950K .......... .......... .......... .......... .......... 4% 166M 1s 2019-12-09T11:38:37.002 INFO:teuthology.orchestra.run.smithi174.stderr: 3000K .......... .......... .......... .......... .......... 4% 186M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3050K .......... .......... .......... .......... .......... 4% 179M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3100K .......... .......... .......... .......... .......... 4% 187M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3150K .......... .......... .......... .......... .......... 4% 154M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3200K .......... .......... .......... .......... .......... 4% 124M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3250K .......... .......... .......... .......... .......... 4% 131M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3300K .......... .......... .......... .......... .......... 4% 123M 1s 2019-12-09T11:38:37.003 INFO:teuthology.orchestra.run.smithi174.stderr: 3350K .......... .......... .......... .......... .......... 4% 113M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3400K .......... .......... .......... .......... .......... 4% 130M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3450K .......... .......... .......... .......... .......... 4% 119M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3500K .......... .......... .......... .......... .......... 4% 126M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3550K .......... .......... .......... .......... .......... 4% 114M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3600K .......... .......... .......... .......... .......... 4% 135M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3650K .......... .......... .......... .......... .......... 5% 134M 1s 2019-12-09T11:38:37.004 INFO:teuthology.orchestra.run.smithi174.stderr: 3700K .......... .......... .......... .......... .......... 5% 139M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 3750K .......... .......... .......... .......... .......... 5% 124M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 3800K .......... .......... .......... .......... .......... 5% 142M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 3850K .......... .......... .......... .......... .......... 5% 140M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 3900K .......... .......... .......... .......... .......... 5% 140M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 3950K .......... .......... .......... .......... .......... 5% 113M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 4000K .......... .......... .......... .......... .......... 5% 136M 1s 2019-12-09T11:38:37.005 INFO:teuthology.orchestra.run.smithi174.stderr: 4050K .......... .......... .......... .......... .......... 5% 131M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4100K .......... .......... .......... .......... .......... 5% 134M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4150K .......... .......... .......... .......... .......... 5% 125M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4200K .......... .......... .......... .......... .......... 5% 140M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4250K .......... .......... .......... .......... .......... 5% 138M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4300K .......... .......... .......... .......... .......... 5% 137M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4350K .......... .......... .......... .......... .......... 5% 109M 1s 2019-12-09T11:38:37.006 INFO:teuthology.orchestra.run.smithi174.stderr: 4400K .......... .......... .......... .......... .......... 6% 136M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4450K .......... .......... .......... .......... .......... 6% 138M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4500K .......... .......... .......... .......... .......... 6% 135M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4550K .......... .......... .......... .......... .......... 6% 126M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4600K .......... .......... .......... .......... .......... 6% 144M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4650K .......... .......... .......... .......... .......... 6% 145M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4700K .......... .......... .......... .......... .......... 6% 129M 1s 2019-12-09T11:38:37.007 INFO:teuthology.orchestra.run.smithi174.stderr: 4750K .......... .......... .......... .......... .......... 6% 108M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 4800K .......... .......... .......... .......... .......... 6% 126M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 4850K .......... .......... .......... .......... .......... 6% 151M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 4900K .......... .......... .......... .......... .......... 6% 137M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 4950K .......... .......... .......... .......... .......... 6% 117M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 5000K .......... .......... .......... .......... .......... 6% 136M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 5050K .......... .......... .......... .......... .......... 6% 138M 1s 2019-12-09T11:38:37.008 INFO:teuthology.orchestra.run.smithi174.stderr: 5100K .......... .......... .......... .......... .......... 7% 140M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5150K .......... .......... .......... .......... .......... 7% 117M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5200K .......... .......... .......... .......... .......... 7% 142M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5250K .......... .......... .......... .......... .......... 7% 142M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5300K .......... .......... .......... .......... .......... 7% 144M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5350K .......... .......... .......... .......... .......... 7% 122M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5400K .......... .......... .......... .......... .......... 7% 149M 1s 2019-12-09T11:38:37.009 INFO:teuthology.orchestra.run.smithi174.stderr: 5450K .......... .......... .......... .......... .......... 7% 141M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5500K .......... .......... .......... .......... .......... 7% 147M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5550K .......... .......... .......... .......... .......... 7% 135M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5600K .......... .......... .......... .......... .......... 7% 137M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5650K .......... .......... .......... .......... .......... 7% 146M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5700K .......... .......... .......... .......... .......... 7% 153M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5750K .......... .......... .......... .......... .......... 7% 130M 1s 2019-12-09T11:38:37.010 INFO:teuthology.orchestra.run.smithi174.stderr: 5800K .......... .......... .......... .......... .......... 7% 153M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 5850K .......... .......... .......... .......... .......... 8% 145M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 5900K .......... .......... .......... .......... .......... 8% 146M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 5950K .......... .......... .......... .......... .......... 8% 124M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 6000K .......... .......... .......... .......... .......... 8% 137M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 6050K .......... .......... .......... .......... .......... 8% 136M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 6100K .......... .......... .......... .......... .......... 8% 150M 1s 2019-12-09T11:38:37.011 INFO:teuthology.orchestra.run.smithi174.stderr: 6150K .......... .......... .......... .......... .......... 8% 128M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6200K .......... .......... .......... .......... .......... 8% 141M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6250K .......... .......... .......... .......... .......... 8% 152M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6300K .......... .......... .......... .......... .......... 8% 154M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6350K .......... .......... .......... .......... .......... 8% 134M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6400K .......... .......... .......... .......... .......... 8% 156M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6450K .......... .......... .......... .......... .......... 8% 158M 1s 2019-12-09T11:38:37.012 INFO:teuthology.orchestra.run.smithi174.stderr: 6500K .......... .......... .......... .......... .......... 8% 148M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6550K .......... .......... .......... .......... .......... 8% 123M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6600K .......... .......... .......... .......... .......... 9% 161M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6650K .......... .......... .......... .......... .......... 9% 148M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6700K .......... .......... .......... .......... .......... 9% 154M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6750K .......... .......... .......... .......... .......... 9% 121M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6800K .......... .......... .......... .......... .......... 9% 145M 1s 2019-12-09T11:38:37.013 INFO:teuthology.orchestra.run.smithi174.stderr: 6850K .......... .......... .......... .......... .......... 9% 153M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 6900K .......... .......... .......... .......... .......... 9% 155M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 6950K .......... .......... .......... .......... .......... 9% 138M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 7000K .......... .......... .......... .......... .......... 9% 157M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 7050K .......... .......... .......... .......... .......... 9% 151M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 7100K .......... .......... .......... .......... .......... 9% 161M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 7150K .......... .......... .......... .......... .......... 9% 130M 1s 2019-12-09T11:38:37.014 INFO:teuthology.orchestra.run.smithi174.stderr: 7200K .......... .......... .......... .......... .......... 9% 154M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7250K .......... .......... .......... .......... .......... 9% 156M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7300K .......... .......... .......... .......... .......... 9% 157M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7350K .......... .......... .......... .......... .......... 10% 139M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7400K .......... .......... .......... .......... .......... 10% 158M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7450K .......... .......... .......... .......... .......... 10% 155M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7500K .......... .......... .......... .......... .......... 10% 162M 1s 2019-12-09T11:38:37.015 INFO:teuthology.orchestra.run.smithi174.stderr: 7550K .......... .......... .......... .......... .......... 10% 126M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7600K .......... .......... .......... .......... .......... 10% 155M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7650K .......... .......... .......... .......... .......... 10% 162M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7700K .......... .......... .......... .......... .......... 10% 145M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7750K .......... .......... .......... .......... .......... 10% 126M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7800K .......... .......... .......... .......... .......... 10% 159M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7850K .......... .......... .......... .......... .......... 10% 158M 1s 2019-12-09T11:38:37.016 INFO:teuthology.orchestra.run.smithi174.stderr: 7900K .......... .......... .......... .......... .......... 10% 153M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 7950K .......... .......... .......... .......... .......... 10% 146M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8000K .......... .......... .......... .......... .......... 10% 153M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8050K .......... .......... .......... .......... .......... 11% 158M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8100K .......... .......... .......... .......... .......... 11% 151M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8150K .......... .......... .......... .......... .......... 11% 136M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8200K .......... .......... .......... .......... .......... 11% 153M 1s 2019-12-09T11:38:37.017 INFO:teuthology.orchestra.run.smithi174.stderr: 8250K .......... .......... .......... .......... .......... 11% 166M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8300K .......... .......... .......... .......... .......... 11% 158M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8350K .......... .......... .......... .......... .......... 11% 136M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8400K .......... .......... .......... .......... .......... 11% 172M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8450K .......... .......... .......... .......... .......... 11% 155M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8500K .......... .......... .......... .......... .......... 11% 166M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8550K .......... .......... .......... .......... .......... 11% 131M 1s 2019-12-09T11:38:37.018 INFO:teuthology.orchestra.run.smithi174.stderr: 8600K .......... .......... .......... .......... .......... 11% 149M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8650K .......... .......... .......... .......... .......... 11% 153M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8700K .......... .......... .......... .......... .......... 11% 153M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8750K .......... .......... .......... .......... .......... 11% 142M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8800K .......... .......... .......... .......... .......... 12% 165M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8850K .......... .......... .......... .......... .......... 12% 163M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8900K .......... .......... .......... .......... .......... 12% 156M 1s 2019-12-09T11:38:37.019 INFO:teuthology.orchestra.run.smithi174.stderr: 8950K .......... .......... .......... .......... .......... 12% 143M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9000K .......... .......... .......... .......... .......... 12% 163M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9050K .......... .......... .......... .......... .......... 12% 173M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9100K .......... .......... .......... .......... .......... 12% 155M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9150K .......... .......... .......... .......... .......... 12% 137M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9200K .......... .......... .......... .......... .......... 12% 162M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9250K .......... .......... .......... .......... .......... 12% 155M 1s 2019-12-09T11:38:37.020 INFO:teuthology.orchestra.run.smithi174.stderr: 9300K .......... .......... .......... .......... .......... 12% 161M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9350K .......... .......... .......... .......... .......... 12% 136M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9400K .......... .......... .......... .......... .......... 12% 149M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9450K .......... .......... .......... .......... .......... 12% 154M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9500K .......... .......... .......... .......... .......... 12% 170M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9550K .......... .......... .......... .......... .......... 13% 154M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9600K .......... .......... .......... .......... .......... 13% 173M 1s 2019-12-09T11:38:37.021 INFO:teuthology.orchestra.run.smithi174.stderr: 9650K .......... .......... .......... .......... .......... 13% 177M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9700K .......... .......... .......... .......... .......... 13% 161M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9750K .......... .......... .......... .......... .......... 13% 147M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9800K .......... .......... .......... .......... .......... 13% 171M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9850K .......... .......... .......... .......... .......... 13% 169M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9900K .......... .......... .......... .......... .......... 13% 156M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 9950K .......... .......... .......... .......... .......... 13% 158M 1s 2019-12-09T11:38:37.022 INFO:teuthology.orchestra.run.smithi174.stderr: 10000K .......... .......... .......... .......... .......... 13% 175M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10050K .......... .......... .......... .......... .......... 13% 158M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10100K .......... .......... .......... .......... .......... 13% 176M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10150K .......... .......... .......... .......... .......... 13% 140M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10200K .......... .......... .......... .......... .......... 13% 169M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10250K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10300K .......... .......... .......... .......... .......... 14% 167M 1s 2019-12-09T11:38:37.023 INFO:teuthology.orchestra.run.smithi174.stderr: 10350K .......... .......... .......... .......... .......... 14% 141M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10400K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10450K .......... .......... .......... .......... .......... 14% 173M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10500K .......... .......... .......... .......... .......... 14% 167M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10550K .......... .......... .......... .......... .......... 14% 134M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10600K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10650K .......... .......... .......... .......... .......... 14% 169M 1s 2019-12-09T11:38:37.024 INFO:teuthology.orchestra.run.smithi174.stderr: 10700K .......... .......... .......... .......... .......... 14% 164M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 10750K .......... .......... .......... .......... .......... 14% 138M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 10800K .......... .......... .......... .......... .......... 14% 164M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 10850K .......... .......... .......... .......... .......... 14% 168M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 10900K .......... .......... .......... .......... .......... 14% 181M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 10950K .......... .......... .......... .......... .......... 14% 131M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 11000K .......... .......... .......... .......... .......... 15% 170M 1s 2019-12-09T11:38:37.025 INFO:teuthology.orchestra.run.smithi174.stderr: 11050K .......... .......... .......... .......... .......... 15% 166M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11100K .......... .......... .......... .......... .......... 15% 171M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11150K .......... .......... .......... .......... .......... 15% 164M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11200K .......... .......... .......... .......... .......... 15% 175M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11250K .......... .......... .......... .......... .......... 15% 171M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11300K .......... .......... .......... .......... .......... 15% 187M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11350K .......... .......... .......... .......... .......... 15% 148M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11400K .......... .......... .......... .......... .......... 15% 175M 1s 2019-12-09T11:38:37.026 INFO:teuthology.orchestra.run.smithi174.stderr: 11450K .......... .......... .......... .......... .......... 15% 173M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11500K .......... .......... .......... .......... .......... 15% 175M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11550K .......... .......... .......... .......... .......... 15% 159M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11600K .......... .......... .......... .......... .......... 15% 178M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11650K .......... .......... .......... .......... .......... 15% 182M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11700K .......... .......... .......... .......... .......... 15% 187M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11750K .......... .......... .......... .......... .......... 16% 144M 1s 2019-12-09T11:38:37.027 INFO:teuthology.orchestra.run.smithi174.stderr: 11800K .......... .......... .......... .......... .......... 16% 169M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 11850K .......... .......... .......... .......... .......... 16% 169M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 11900K .......... .......... .......... .......... .......... 16% 169M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 11950K .......... .......... .......... .......... .......... 16% 153M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 12000K .......... .......... .......... .......... .......... 16% 170M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 12050K .......... .......... .......... .......... .......... 16% 184M 1s 2019-12-09T11:38:37.028 INFO:teuthology.orchestra.run.smithi174.stderr: 12100K .......... .......... .......... .......... .......... 16% 184M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12150K .......... .......... .......... .......... .......... 16% 149M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12200K .......... .......... .......... .......... .......... 16% 179M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12250K .......... .......... .......... .......... .......... 16% 184M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12300K .......... .......... .......... .......... .......... 16% 181M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12350K .......... .......... .......... .......... .......... 16% 150M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12400K .......... .......... .......... .......... .......... 16% 170M 1s 2019-12-09T11:38:37.029 INFO:teuthology.orchestra.run.smithi174.stderr: 12450K .......... .......... .......... .......... .......... 16% 176M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12500K .......... .......... .......... .......... .......... 17% 163M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12550K .......... .......... .......... .......... .......... 17% 155M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12600K .......... .......... .......... .......... .......... 17% 171M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12650K .......... .......... .......... .......... .......... 17% 176M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12700K .......... .......... .......... .......... .......... 17% 179M 1s 2019-12-09T11:38:37.030 INFO:teuthology.orchestra.run.smithi174.stderr: 12750K .......... .......... .......... .......... .......... 17% 145M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 12800K .......... .......... .......... .......... .......... 17% 181M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 12850K .......... .......... .......... .......... .......... 17% 172M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 12900K .......... .......... .......... .......... .......... 17% 167M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 12950K .......... .......... .......... .......... .......... 17% 159M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 13000K .......... .......... .......... .......... .......... 17% 177M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 13050K .......... .......... .......... .......... .......... 17% 170M 1s 2019-12-09T11:38:37.031 INFO:teuthology.orchestra.run.smithi174.stderr: 13100K .......... .......... .......... .......... .......... 17% 220M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13150K .......... .......... .......... .......... .......... 17% 160M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13200K .......... .......... .......... .......... .......... 18% 189M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13250K .......... .......... .......... .......... .......... 18% 240M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13300K .......... .......... .......... .......... .......... 18% 183M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13350K .......... .......... .......... .......... .......... 18% 212M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13400K .......... .......... .......... .......... .......... 18% 266M 1s 2019-12-09T11:38:37.032 INFO:teuthology.orchestra.run.smithi174.stderr: 13450K .......... .......... .......... .......... .......... 18% 256M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13500K .......... .......... .......... .......... .......... 18% 240M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13550K .......... .......... .......... .......... .......... 18% 205M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13600K .......... .......... .......... .......... .......... 18% 201M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13650K .......... .......... .......... .......... .......... 18% 228M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13700K .......... .......... .......... .......... .......... 18% 192M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13750K .......... .......... .......... .......... .......... 18% 187M 1s 2019-12-09T11:38:37.033 INFO:teuthology.orchestra.run.smithi174.stderr: 13800K .......... .......... .......... .......... .......... 18% 264M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 13850K .......... .......... .......... .......... .......... 18% 251M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 13900K .......... .......... .......... .......... .......... 18% 247M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 13950K .......... .......... .......... .......... .......... 19% 201M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 14000K .......... .......... .......... .......... .......... 19% 235M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 14050K .......... .......... .......... .......... .......... 19% 229M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 14100K .......... .......... .......... .......... .......... 19% 214M 0s 2019-12-09T11:38:37.034 INFO:teuthology.orchestra.run.smithi174.stderr: 14150K .......... .......... .......... .......... .......... 19% 204M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14200K .......... .......... .......... .......... .......... 19% 253M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14250K .......... .......... .......... .......... .......... 19% 252M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14300K .......... .......... .......... .......... .......... 19% 243M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14350K .......... .......... .......... .......... .......... 19% 219M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14400K .......... .......... .......... .......... .......... 19% 237M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14450K .......... .......... .......... .......... .......... 19% 246M 0s 2019-12-09T11:38:37.035 INFO:teuthology.orchestra.run.smithi174.stderr: 14500K .......... .......... .......... .......... .......... 19% 245M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14550K .......... .......... .......... .......... .......... 19% 216M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14600K .......... .......... .......... .......... .......... 19% 253M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14650K .......... .......... .......... .......... .......... 19% 257M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14700K .......... .......... .......... .......... .......... 20% 237M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14750K .......... .......... .......... .......... .......... 20% 203M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14800K .......... .......... .......... .......... .......... 20% 249M 0s 2019-12-09T11:38:37.036 INFO:teuthology.orchestra.run.smithi174.stderr: 14850K .......... .......... .......... .......... .......... 20% 252M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 14900K .......... .......... .......... .......... .......... 20% 245M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 14950K .......... .......... .......... .......... .......... 20% 223M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 15000K .......... .......... .......... .......... .......... 20% 257M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 15050K .......... .......... .......... .......... .......... 20% 261M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 15100K .......... .......... .......... .......... .......... 20% 258M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 15150K .......... .......... .......... .......... .......... 20% 225M 0s 2019-12-09T11:38:37.037 INFO:teuthology.orchestra.run.smithi174.stderr: 15200K .......... .......... .......... .......... .......... 20% 258M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15250K .......... .......... .......... .......... .......... 20% 251M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15300K .......... .......... .......... .......... .......... 20% 252M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15350K .......... .......... .......... .......... .......... 20% 215M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15400K .......... .......... .......... .......... .......... 21% 251M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15450K .......... .......... .......... .......... .......... 21% 246M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15500K .......... .......... .......... .......... .......... 21% 231M 0s 2019-12-09T11:38:37.038 INFO:teuthology.orchestra.run.smithi174.stderr: 15550K .......... .......... .......... .......... .......... 21% 237M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15600K .......... .......... .......... .......... .......... 21% 259M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15650K .......... .......... .......... .......... .......... 21% 260M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15700K .......... .......... .......... .......... .......... 21% 235M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15750K .......... .......... .......... .......... .......... 21% 223M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15800K .......... .......... .......... .......... .......... 21% 264M 0s 2019-12-09T11:38:37.039 INFO:teuthology.orchestra.run.smithi174.stderr: 15850K .......... .......... .......... .......... .......... 21% 247M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 15900K .......... .......... .......... .......... .......... 21% 250M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 15950K .......... .......... .......... .......... .......... 21% 203M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 16000K .......... .......... .......... .......... .......... 21% 235M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 16050K .......... .......... .......... .......... .......... 21% 253M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 16100K .......... .......... .......... .......... .......... 21% 263M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 16150K .......... .......... .......... .......... .......... 22% 231M 0s 2019-12-09T11:38:37.040 INFO:teuthology.orchestra.run.smithi174.stderr: 16200K .......... .......... .......... .......... .......... 22% 229M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16250K .......... .......... .......... .......... .......... 22% 267M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16300K .......... .......... .......... .......... .......... 22% 271M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16350K .......... .......... .......... .......... .......... 22% 226M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16400K .......... .......... .......... .......... .......... 22% 264M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16450K .......... .......... .......... .......... .......... 22% 246M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16500K .......... .......... .......... .......... .......... 22% 264M 0s 2019-12-09T11:38:37.041 INFO:teuthology.orchestra.run.smithi174.stderr: 16550K .......... .......... .......... .......... .......... 22% 222M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16600K .......... .......... .......... .......... .......... 22% 261M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16650K .......... .......... .......... .......... .......... 22% 270M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16700K .......... .......... .......... .......... .......... 22% 256M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16750K .......... .......... .......... .......... .......... 22% 235M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16800K .......... .......... .......... .......... .......... 22% 274M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16850K .......... .......... .......... .......... .......... 22% 269M 0s 2019-12-09T11:38:37.042 INFO:teuthology.orchestra.run.smithi174.stderr: 16900K .......... .......... .......... .......... .......... 23% 271M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 16950K .......... .......... .......... .......... .......... 23% 220M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17000K .......... .......... .......... .......... .......... 23% 268M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17050K .......... .......... .......... .......... .......... 23% 264M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17100K .......... .......... .......... .......... .......... 23% 264M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17150K .......... .......... .......... .......... .......... 23% 234M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17200K .......... .......... .......... .......... .......... 23% 278M 0s 2019-12-09T11:38:37.043 INFO:teuthology.orchestra.run.smithi174.stderr: 17250K .......... .......... .......... .......... .......... 23% 266M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17300K .......... .......... .......... .......... .......... 23% 282M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17350K .......... .......... .......... .......... .......... 23% 215M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17400K .......... .......... .......... .......... .......... 23% 255M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17450K .......... .......... .......... .......... .......... 23% 276M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17500K .......... .......... .......... .......... .......... 23% 269M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17550K .......... .......... .......... .......... .......... 23% 234M 0s 2019-12-09T11:38:37.044 INFO:teuthology.orchestra.run.smithi174.stderr: 17600K .......... .......... .......... .......... .......... 23% 259M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17650K .......... .......... .......... .......... .......... 24% 274M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17700K .......... .......... .......... .......... .......... 24% 255M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17750K .......... .......... .......... .......... .......... 24% 196M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17800K .......... .......... .......... .......... .......... 24% 260M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17850K .......... .......... .......... .......... .......... 24% 241M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17900K .......... .......... .......... .......... .......... 24% 269M 0s 2019-12-09T11:38:37.045 INFO:teuthology.orchestra.run.smithi174.stderr: 17950K .......... .......... .......... .......... .......... 24% 224M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18000K .......... .......... .......... .......... .......... 24% 221M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18050K .......... .......... .......... .......... .......... 24% 244M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18100K .......... .......... .......... .......... .......... 24% 248M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18150K .......... .......... .......... .......... .......... 24% 222M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18200K .......... .......... .......... .......... .......... 24% 232M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18250K .......... .......... .......... .......... .......... 24% 254M 0s 2019-12-09T11:38:37.046 INFO:teuthology.orchestra.run.smithi174.stderr: 18300K .......... .......... .......... .......... .......... 24% 261M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18350K .......... .......... .......... .......... .......... 25% 221M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18400K .......... .......... .......... .......... .......... 25% 282M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18450K .......... .......... .......... .......... .......... 25% 279M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18500K .......... .......... .......... .......... .......... 25% 246M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18550K .......... .......... .......... .......... .......... 25% 234M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18600K .......... .......... .......... .......... .......... 25% 254M 0s 2019-12-09T11:38:37.047 INFO:teuthology.orchestra.run.smithi174.stderr: 18650K .......... .......... .......... .......... .......... 25% 290M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18700K .......... .......... .......... .......... .......... 25% 284M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18750K .......... .......... .......... .......... .......... 25% 229M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18800K .......... .......... .......... .......... .......... 25% 262M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18850K .......... .......... .......... .......... .......... 25% 295M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18900K .......... .......... .......... .......... .......... 25% 273M 0s 2019-12-09T11:38:37.048 INFO:teuthology.orchestra.run.smithi174.stderr: 18950K .......... .......... .......... .......... .......... 25% 252M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19000K .......... .......... .......... .......... .......... 25% 258M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19050K .......... .......... .......... .......... .......... 25% 273M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19100K .......... .......... .......... .......... .......... 26% 263M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19150K .......... .......... .......... .......... .......... 26% 248M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19200K .......... .......... .......... .......... .......... 26% 273M 0s 2019-12-09T11:38:37.049 INFO:teuthology.orchestra.run.smithi174.stderr: 19250K .......... .......... .......... .......... .......... 26% 284M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19300K .......... .......... .......... .......... .......... 26% 267M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19350K .......... .......... .......... .......... .......... 26% 230M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19400K .......... .......... .......... .......... .......... 26% 294M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19450K .......... .......... .......... .......... .......... 26% 266M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19500K .......... .......... .......... .......... .......... 26% 284M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19550K .......... .......... .......... .......... .......... 26% 260M 0s 2019-12-09T11:38:37.050 INFO:teuthology.orchestra.run.smithi174.stderr: 19600K .......... .......... .......... .......... .......... 26% 269M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19650K .......... .......... .......... .......... .......... 26% 264M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19700K .......... .......... .......... .......... .......... 26% 289M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19750K .......... .......... .......... .......... .......... 26% 234M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19800K .......... .......... .......... .......... .......... 26% 297M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19850K .......... .......... .......... .......... .......... 27% 289M 0s 2019-12-09T11:38:37.051 INFO:teuthology.orchestra.run.smithi174.stderr: 19900K .......... .......... .......... .......... .......... 27% 292M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 19950K .......... .......... .......... .......... .......... 27% 249M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20000K .......... .......... .......... .......... .......... 27% 296M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20050K .......... .......... .......... .......... .......... 27% 283M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20100K .......... .......... .......... .......... .......... 27% 290M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20150K .......... .......... .......... .......... .......... 27% 242M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20200K .......... .......... .......... .......... .......... 27% 267M 0s 2019-12-09T11:38:37.052 INFO:teuthology.orchestra.run.smithi174.stderr: 20250K .......... .......... .......... .......... .......... 27% 287M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20300K .......... .......... .......... .......... .......... 27% 295M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20350K .......... .......... .......... .......... .......... 27% 260M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20400K .......... .......... .......... .......... .......... 27% 290M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20450K .......... .......... .......... .......... .......... 27% 281M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20500K .......... .......... .......... .......... .......... 27% 300M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20550K .......... .......... .......... .......... .......... 28% 250M 0s 2019-12-09T11:38:37.053 INFO:teuthology.orchestra.run.smithi174.stderr: 20600K .......... .......... .......... .......... .......... 28% 297M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20650K .......... .......... .......... .......... .......... 28% 304M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20700K .......... .......... .......... .......... .......... 28% 293M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20750K .......... .......... .......... .......... .......... 28% 263M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20800K .......... .......... .......... .......... .......... 28% 284M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20850K .......... .......... .......... .......... .......... 28% 281M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20900K .......... .......... .......... .......... .......... 28% 262M 0s 2019-12-09T11:38:37.054 INFO:teuthology.orchestra.run.smithi174.stderr: 20950K .......... .......... .......... .......... .......... 28% 226M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21000K .......... .......... .......... .......... .......... 28% 270M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21050K .......... .......... .......... .......... .......... 28% 272M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21100K .......... .......... .......... .......... .......... 28% 272M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21150K .......... .......... .......... .......... .......... 28% 234M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21200K .......... .......... .......... .......... .......... 28% 283M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21250K .......... .......... .......... .......... .......... 28% 265M 0s 2019-12-09T11:38:37.055 INFO:teuthology.orchestra.run.smithi174.stderr: 21300K .......... .......... .......... .......... .......... 29% 279M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21350K .......... .......... .......... .......... .......... 29% 221M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21400K .......... .......... .......... .......... .......... 29% 290M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21450K .......... .......... .......... .......... .......... 29% 299M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21500K .......... .......... .......... .......... .......... 29% 297M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21550K .......... .......... .......... .......... .......... 29% 248M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21600K .......... .......... .......... .......... .......... 29% 297M 0s 2019-12-09T11:38:37.056 INFO:teuthology.orchestra.run.smithi174.stderr: 21650K .......... .......... .......... .......... .......... 29% 285M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21700K .......... .......... .......... .......... .......... 29% 287M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21750K .......... .......... .......... .......... .......... 29% 229M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21800K .......... .......... .......... .......... .......... 29% 287M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21850K .......... .......... .......... .......... .......... 29% 304M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21900K .......... .......... .......... .......... .......... 29% 313M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 21950K .......... .......... .......... .......... .......... 29% 254M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 22000K .......... .......... .......... .......... .......... 29% 310M 0s 2019-12-09T11:38:37.057 INFO:teuthology.orchestra.run.smithi174.stderr: 22050K .......... .......... .......... .......... .......... 30% 306M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22100K .......... .......... .......... .......... .......... 30% 307M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22150K .......... .......... .......... .......... .......... 30% 264M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22200K .......... .......... .......... .......... .......... 30% 295M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22250K .......... .......... .......... .......... .......... 30% 293M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22300K .......... .......... .......... .......... .......... 30% 285M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22350K .......... .......... .......... .......... .......... 30% 251M 0s 2019-12-09T11:38:37.058 INFO:teuthology.orchestra.run.smithi174.stderr: 22400K .......... .......... .......... .......... .......... 30% 294M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22450K .......... .......... .......... .......... .......... 30% 287M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22500K .......... .......... .......... .......... .......... 30% 285M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22550K .......... .......... .......... .......... .......... 30% 245M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22600K .......... .......... .......... .......... .......... 30% 263M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22650K .......... .......... .......... .......... .......... 30% 281M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22700K .......... .......... .......... .......... .......... 30% 273M 0s 2019-12-09T11:38:37.059 INFO:teuthology.orchestra.run.smithi174.stderr: 22750K .......... .......... .......... .......... .......... 30% 256M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 22800K .......... .......... .......... .......... .......... 31% 292M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 22850K .......... .......... .......... .......... .......... 31% 291M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 22900K .......... .......... .......... .......... .......... 31% 309M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 22950K .......... .......... .......... .......... .......... 31% 248M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 23000K .......... .......... .......... .......... .......... 31% 286M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 23050K .......... .......... .......... .......... .......... 31% 283M 0s 2019-12-09T11:38:37.060 INFO:teuthology.orchestra.run.smithi174.stderr: 23100K .......... .......... .......... .......... .......... 31% 293M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23150K .......... .......... .......... .......... .......... 31% 250M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23200K .......... .......... .......... .......... .......... 31% 290M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23250K .......... .......... .......... .......... .......... 31% 301M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23300K .......... .......... .......... .......... .......... 31% 297M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23350K .......... .......... .......... .......... .......... 31% 237M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23400K .......... .......... .......... .......... .......... 31% 283M 0s 2019-12-09T11:38:37.061 INFO:teuthology.orchestra.run.smithi174.stderr: 23450K .......... .......... .......... .......... .......... 31% 298M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23500K .......... .......... .......... .......... .......... 32% 279M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23550K .......... .......... .......... .......... .......... 32% 239M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23600K .......... .......... .......... .......... .......... 32% 298M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23650K .......... .......... .......... .......... .......... 32% 293M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23700K .......... .......... .......... .......... .......... 32% 274M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23750K .......... .......... .......... .......... .......... 32% 257M 0s 2019-12-09T11:38:37.062 INFO:teuthology.orchestra.run.smithi174.stderr: 23800K .......... .......... .......... .......... .......... 32% 266M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 23850K .......... .......... .......... .......... .......... 32% 296M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 23900K .......... .......... .......... .......... .......... 32% 269M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 23950K .......... .......... .......... .......... .......... 32% 226M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 24000K .......... .......... .......... .......... .......... 32% 294M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 24050K .......... .......... .......... .......... .......... 32% 300M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 24100K .......... .......... .......... .......... .......... 32% 265M 0s 2019-12-09T11:38:37.063 INFO:teuthology.orchestra.run.smithi174.stderr: 24150K .......... .......... .......... .......... .......... 32% 228M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24200K .......... .......... .......... .......... .......... 32% 301M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24250K .......... .......... .......... .......... .......... 33% 294M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24300K .......... .......... .......... .......... .......... 33% 296M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24350K .......... .......... .......... .......... .......... 33% 241M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24400K .......... .......... .......... .......... .......... 33% 290M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24450K .......... .......... .......... .......... .......... 33% 292M 0s 2019-12-09T11:38:37.064 INFO:teuthology.orchestra.run.smithi174.stderr: 24500K .......... .......... .......... .......... .......... 33% 308M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24550K .......... .......... .......... .......... .......... 33% 255M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24600K .......... .......... .......... .......... .......... 33% 309M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24650K .......... .......... .......... .......... .......... 33% 320M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24700K .......... .......... .......... .......... .......... 33% 300M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24750K .......... .......... .......... .......... .......... 33% 260M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24800K .......... .......... .......... .......... .......... 33% 307M 0s 2019-12-09T11:38:37.065 INFO:teuthology.orchestra.run.smithi174.stderr: 24850K .......... .......... .......... .......... .......... 33% 314M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 24900K .......... .......... .......... .......... .......... 33% 307M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 24950K .......... .......... .......... .......... .......... 33% 258M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 25000K .......... .......... .......... .......... .......... 34% 314M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 25050K .......... .......... .......... .......... .......... 34% 320M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 25100K .......... .......... .......... .......... .......... 34% 308M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 25150K .......... .......... .......... .......... .......... 34% 273M 0s 2019-12-09T11:38:37.066 INFO:teuthology.orchestra.run.smithi174.stderr: 25200K .......... .......... .......... .......... .......... 34% 310M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25250K .......... .......... .......... .......... .......... 34% 319M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25300K .......... .......... .......... .......... .......... 34% 313M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25350K .......... .......... .......... .......... .......... 34% 265M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25400K .......... .......... .......... .......... .......... 34% 294M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25450K .......... .......... .......... .......... .......... 34% 294M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25500K .......... .......... .......... .......... .......... 34% 297M 0s 2019-12-09T11:38:37.067 INFO:teuthology.orchestra.run.smithi174.stderr: 25550K .......... .......... .......... .......... .......... 34% 245M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25600K .......... .......... .......... .......... .......... 34% 289M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25650K .......... .......... .......... .......... .......... 34% 307M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25700K .......... .......... .......... .......... .......... 35% 320M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25750K .......... .......... .......... .......... .......... 35% 246M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25800K .......... .......... .......... .......... .......... 35% 308M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25850K .......... .......... .......... .......... .......... 35% 294M 0s 2019-12-09T11:38:37.068 INFO:teuthology.orchestra.run.smithi174.stderr: 25900K .......... .......... .......... .......... .......... 35% 295M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 25950K .......... .......... .......... .......... .......... 35% 267M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26000K .......... .......... .......... .......... .......... 35% 316M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26050K .......... .......... .......... .......... .......... 35% 324M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26100K .......... .......... .......... .......... .......... 35% 321M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26150K .......... .......... .......... .......... .......... 35% 283M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26200K .......... .......... .......... .......... .......... 35% 326M 0s 2019-12-09T11:38:37.069 INFO:teuthology.orchestra.run.smithi174.stderr: 26250K .......... .......... .......... .......... .......... 35% 308M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26300K .......... .......... .......... .......... .......... 35% 325M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26350K .......... .......... .......... .......... .......... 35% 275M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26400K .......... .......... .......... .......... .......... 35% 325M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26450K .......... .......... .......... .......... .......... 36% 325M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26500K .......... .......... .......... .......... .......... 36% 317M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26550K .......... .......... .......... .......... .......... 36% 281M 0s 2019-12-09T11:38:37.070 INFO:teuthology.orchestra.run.smithi174.stderr: 26600K .......... .......... .......... .......... .......... 36% 308M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26650K .......... .......... .......... .......... .......... 36% 326M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26700K .......... .......... .......... .......... .......... 36% 326M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26750K .......... .......... .......... .......... .......... 36% 282M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26800K .......... .......... .......... .......... .......... 36% 303M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26850K .......... .......... .......... .......... .......... 36% 315M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26900K .......... .......... .......... .......... .......... 36% 323M 0s 2019-12-09T11:38:37.071 INFO:teuthology.orchestra.run.smithi174.stderr: 26950K .......... .......... .......... .......... .......... 36% 266M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27000K .......... .......... .......... .......... .......... 36% 329M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27050K .......... .......... .......... .......... .......... 36% 316M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27100K .......... .......... .......... .......... .......... 36% 278M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27150K .......... .......... .......... .......... .......... 36% 256M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27200K .......... .......... .......... .......... .......... 37% 336M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27250K .......... .......... .......... .......... .......... 37% 336M 0s 2019-12-09T11:38:37.072 INFO:teuthology.orchestra.run.smithi174.stderr: 27300K .......... .......... .......... .......... .......... 37% 334M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27350K .......... .......... .......... .......... .......... 37% 274M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27400K .......... .......... .......... .......... .......... 37% 323M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27450K .......... .......... .......... .......... .......... 37% 161M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27500K .......... .......... .......... .......... .......... 37% 216M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27550K .......... .......... .......... .......... .......... 37% 219M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27600K .......... .......... .......... .......... .......... 37% 44.4M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27650K .......... .......... .......... .......... .......... 37% 207M 0s 2019-12-09T11:38:37.073 INFO:teuthology.orchestra.run.smithi174.stderr: 27700K .......... .......... .......... .......... .......... 37% 214M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 27750K .......... .......... .......... .......... .......... 37% 229M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 27800K .......... .......... .......... .......... .......... 37% 269M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 27850K .......... .......... .......... .......... .......... 37% 279M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 27900K .......... .......... .......... .......... .......... 37% 239M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 27950K .......... .......... .......... .......... .......... 38% 235M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 28000K .......... .......... .......... .......... .......... 38% 262M 0s 2019-12-09T11:38:37.074 INFO:teuthology.orchestra.run.smithi174.stderr: 28050K .......... .......... .......... .......... .......... 38% 251M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28100K .......... .......... .......... .......... .......... 38% 276M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28150K .......... .......... .......... .......... .......... 38% 237M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28200K .......... .......... .......... .......... .......... 38% 238M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28250K .......... .......... .......... .......... .......... 38% 216M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28300K .......... .......... .......... .......... .......... 38% 25.7M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28350K .......... .......... .......... .......... .......... 38% 199M 0s 2019-12-09T11:38:37.075 INFO:teuthology.orchestra.run.smithi174.stderr: 28400K .......... .......... .......... .......... .......... 38% 219M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28450K .......... .......... .......... .......... .......... 38% 209M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28500K .......... .......... .......... .......... .......... 38% 229M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28550K .......... .......... .......... .......... .......... 38% 203M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28600K .......... .......... .......... .......... .......... 38% 245M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28650K .......... .......... .......... .......... .......... 39% 282M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28700K .......... .......... .......... .......... .......... 39% 247M 0s 2019-12-09T11:38:37.076 INFO:teuthology.orchestra.run.smithi174.stderr: 28750K .......... .......... .......... .......... .......... 39% 217M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 28800K .......... .......... .......... .......... .......... 39% 257M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 28850K .......... .......... .......... .......... .......... 39% 357M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 28900K .......... .......... .......... .......... .......... 39% 329M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 28950K .......... .......... .......... .......... .......... 39% 233M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 29000K .......... .......... .......... .......... .......... 39% 230M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 29050K .......... .......... .......... .......... .......... 39% 70.4M 0s 2019-12-09T11:38:37.077 INFO:teuthology.orchestra.run.smithi174.stderr: 29100K .......... .......... .......... .......... .......... 39% 236M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29150K .......... .......... .......... .......... .......... 39% 259M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29200K .......... .......... .......... .......... .......... 39% 23.1M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29250K .......... .......... .......... .......... .......... 39% 315M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29300K .......... .......... .......... .......... .......... 39% 328M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29350K .......... .......... .......... .......... .......... 39% 293M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29400K .......... .......... .......... .......... .......... 40% 317M 0s 2019-12-09T11:38:37.078 INFO:teuthology.orchestra.run.smithi174.stderr: 29450K .......... .......... .......... .......... .......... 40% 320M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29500K .......... .......... .......... .......... .......... 40% 308M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29550K .......... .......... .......... .......... .......... 40% 217M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29600K .......... .......... .......... .......... .......... 40% 323M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29650K .......... .......... .......... .......... .......... 40% 308M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29700K .......... .......... .......... .......... .......... 40% 313M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29750K .......... .......... .......... .......... .......... 40% 268M 0s 2019-12-09T11:38:37.079 INFO:teuthology.orchestra.run.smithi174.stderr: 29800K .......... .......... .......... .......... .......... 40% 266M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 29850K .......... .......... .......... .......... .......... 40% 238M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 29900K .......... .......... .......... .......... .......... 40% 315M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 29950K .......... .......... .......... .......... .......... 40% 233M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 30000K .......... .......... .......... .......... .......... 40% 299M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 30050K .......... .......... .......... .......... .......... 40% 326M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 30100K .......... .......... .......... .......... .......... 40% 319M 0s 2019-12-09T11:38:37.080 INFO:teuthology.orchestra.run.smithi174.stderr: 30150K .......... .......... .......... .......... .......... 41% 286M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30200K .......... .......... .......... .......... .......... 41% 28.0M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30250K .......... .......... .......... .......... .......... 41% 276M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30300K .......... .......... .......... .......... .......... 41% 226M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30350K .......... .......... .......... .......... .......... 41% 176M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30400K .......... .......... .......... .......... .......... 41% 237M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30450K .......... .......... .......... .......... .......... 41% 237M 0s 2019-12-09T11:38:37.081 INFO:teuthology.orchestra.run.smithi174.stderr: 30500K .......... .......... .......... .......... .......... 41% 282M 0s 2019-12-09T11:38:37.082 INFO:teuthology.orchestra.run.smithi174.stderr: 30550K .......... .......... .......... .......... .......... 41% 237M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30600K .......... .......... .......... .......... .......... 41% 283M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30650K .......... .......... .......... .......... .......... 41% 289M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30700K .......... .......... .......... .......... .......... 41% 86.3M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30750K .......... .......... .......... .......... .......... 41% 192M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30800K .......... .......... .......... .......... .......... 41% 230M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30850K .......... .......... .......... .......... .......... 42% 233M 0s 2019-12-09T11:38:37.084 INFO:teuthology.orchestra.run.smithi174.stderr: 30900K .......... .......... .......... .......... .......... 42% 256M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 30950K .......... .......... .......... .......... .......... 42% 56.6M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31000K .......... .......... .......... .......... .......... 42% 271M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31050K .......... .......... .......... .......... .......... 42% 288M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31100K .......... .......... .......... .......... .......... 42% 140M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31150K .......... .......... .......... .......... .......... 42% 267M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31200K .......... .......... .......... .......... .......... 42% 75.6M 0s 2019-12-09T11:38:37.085 INFO:teuthology.orchestra.run.smithi174.stderr: 31250K .......... .......... .......... .......... .......... 42% 303M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31300K .......... .......... .......... .......... .......... 42% 254M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31350K .......... .......... .......... .......... .......... 42% 94.1M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31400K .......... .......... .......... .......... .......... 42% 203M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31450K .......... .......... .......... .......... .......... 42% 225M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31500K .......... .......... .......... .......... .......... 42% 288M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31550K .......... .......... .......... .......... .......... 42% 296M 0s 2019-12-09T11:38:37.086 INFO:teuthology.orchestra.run.smithi174.stderr: 31600K .......... .......... .......... .......... .......... 43% 131M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31650K .......... .......... .......... .......... .......... 43% 124M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31700K .......... .......... .......... .......... .......... 43% 22.8M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31750K .......... .......... .......... .......... .......... 43% 27.7M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31800K .......... .......... .......... .......... .......... 43% 265M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31850K .......... .......... .......... .......... .......... 43% 273M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31900K .......... .......... .......... .......... .......... 43% 277M 0s 2019-12-09T11:38:37.087 INFO:teuthology.orchestra.run.smithi174.stderr: 31950K .......... .......... .......... .......... .......... 43% 47.8M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32000K .......... .......... .......... .......... .......... 43% 256M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32050K .......... .......... .......... .......... .......... 43% 275M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32100K .......... .......... .......... .......... .......... 43% 182M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32150K .......... .......... .......... .......... .......... 43% 195M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32200K .......... .......... .......... .......... .......... 43% 280M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32250K .......... .......... .......... .......... .......... 43% 122M 0s 2019-12-09T11:38:37.088 INFO:teuthology.orchestra.run.smithi174.stderr: 32300K .......... .......... .......... .......... .......... 43% 268M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32350K .......... .......... .......... .......... .......... 44% 226M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32400K .......... .......... .......... .......... .......... 44% 279M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32450K .......... .......... .......... .......... .......... 44% 277M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32500K .......... .......... .......... .......... .......... 44% 93.3M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32550K .......... .......... .......... .......... .......... 44% 242M 0s 2019-12-09T11:38:37.089 INFO:teuthology.orchestra.run.smithi174.stderr: 32600K .......... .......... .......... .......... .......... 44% 401M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32650K .......... .......... .......... .......... .......... 44% 376M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32700K .......... .......... .......... .......... .......... 44% 55.8M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32750K .......... .......... .......... .......... .......... 44% 154M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32800K .......... .......... .......... .......... .......... 44% 310M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32850K .......... .......... .......... .......... .......... 44% 315M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32900K .......... .......... .......... .......... .......... 44% 323M 0s 2019-12-09T11:38:37.090 INFO:teuthology.orchestra.run.smithi174.stderr: 32950K .......... .......... .......... .......... .......... 44% 279M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33000K .......... .......... .......... .......... .......... 44% 71.2M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33050K .......... .......... .......... .......... .......... 44% 299M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33100K .......... .......... .......... .......... .......... 45% 112M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33150K .......... .......... .......... .......... .......... 45% 270M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33200K .......... .......... .......... .......... .......... 45% 218M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33250K .......... .......... .......... .......... .......... 45% 136M 0s 2019-12-09T11:38:37.091 INFO:teuthology.orchestra.run.smithi174.stderr: 33300K .......... .......... .......... .......... .......... 45% 255M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33350K .......... .......... .......... .......... .......... 45% 244M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33400K .......... .......... .......... .......... .......... 45% 31.9M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33450K .......... .......... .......... .......... .......... 45% 192M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33500K .......... .......... .......... .......... .......... 45% 263M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33550K .......... .......... .......... .......... .......... 45% 230M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33600K .......... .......... .......... .......... .......... 45% 278M 0s 2019-12-09T11:38:37.092 INFO:teuthology.orchestra.run.smithi174.stderr: 33650K .......... .......... .......... .......... .......... 45% 273M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33700K .......... .......... .......... .......... .......... 45% 274M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33750K .......... .......... .......... .......... .......... 45% 246M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33800K .......... .......... .......... .......... .......... 46% 266M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33850K .......... .......... .......... .......... .......... 46% 278M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33900K .......... .......... .......... .......... .......... 46% 264M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 33950K .......... .......... .......... .......... .......... 46% 218M 0s 2019-12-09T11:38:37.093 INFO:teuthology.orchestra.run.smithi174.stderr: 34000K .......... .......... .......... .......... .......... 46% 278M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34050K .......... .......... .......... .......... .......... 46% 269M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34100K .......... .......... .......... .......... .......... 46% 281M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34150K .......... .......... .......... .......... .......... 46% 256M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34200K .......... .......... .......... .......... .......... 46% 295M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34250K .......... .......... .......... .......... .......... 46% 267M 0s 2019-12-09T11:38:37.094 INFO:teuthology.orchestra.run.smithi174.stderr: 34300K .......... .......... .......... .......... .......... 46% 276M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34350K .......... .......... .......... .......... .......... 46% 242M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34400K .......... .......... .......... .......... .......... 46% 46.2M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34450K .......... .......... .......... .......... .......... 46% 260M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34500K .......... .......... .......... .......... .......... 46% 283M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34550K .......... .......... .......... .......... .......... 47% 244M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34600K .......... .......... .......... .......... .......... 47% 282M 0s 2019-12-09T11:38:37.095 INFO:teuthology.orchestra.run.smithi174.stderr: 34650K .......... .......... .......... .......... .......... 47% 230M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34700K .......... .......... .......... .......... .......... 47% 384M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34750K .......... .......... .......... .......... .......... 47% 352M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34800K .......... .......... .......... .......... .......... 47% 340M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34850K .......... .......... .......... .......... .......... 47% 258M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34900K .......... .......... .......... .......... .......... 47% 117M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 34950K .......... .......... .......... .......... .......... 47% 174M 0s 2019-12-09T11:38:37.096 INFO:teuthology.orchestra.run.smithi174.stderr: 35000K .......... .......... .......... .......... .......... 47% 210M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35050K .......... .......... .......... .......... .......... 47% 113M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35100K .......... .......... .......... .......... .......... 47% 205M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35150K .......... .......... .......... .......... .......... 47% 74.9M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35200K .......... .......... .......... .......... .......... 47% 92.5M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35250K .......... .......... .......... .......... .......... 47% 214M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35300K .......... .......... .......... .......... .......... 48% 275M 0s 2019-12-09T11:38:37.097 INFO:teuthology.orchestra.run.smithi174.stderr: 35350K .......... .......... .......... .......... .......... 48% 260M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35400K .......... .......... .......... .......... .......... 48% 101M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35450K .......... .......... .......... .......... .......... 48% 205M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35500K .......... .......... .......... .......... .......... 48% 252M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35550K .......... .......... .......... .......... .......... 48% 266M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35600K .......... .......... .......... .......... .......... 48% 354M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35650K .......... .......... .......... .......... .......... 48% 106M 0s 2019-12-09T11:38:37.098 INFO:teuthology.orchestra.run.smithi174.stderr: 35700K .......... .......... .......... .......... .......... 48% 305M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 35750K .......... .......... .......... .......... .......... 48% 323M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 35800K .......... .......... .......... .......... .......... 48% 385M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 35850K .......... .......... .......... .......... .......... 48% 380M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 35900K .......... .......... .......... .......... .......... 48% 405M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 35950K .......... .......... .......... .......... .......... 48% 194M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 36000K .......... .......... .......... .......... .......... 49% 204M 0s 2019-12-09T11:38:37.099 INFO:teuthology.orchestra.run.smithi174.stderr: 36050K .......... .......... .......... .......... .......... 49% 163M 0s 2019-12-09T11:38:37.100 INFO:teuthology.orchestra.run.smithi174.stderr: 36100K .......... .......... .......... .......... .......... 49% 52.5M 0s 2019-12-09T11:38:37.100 INFO:teuthology.orchestra.run.smithi174.stderr: 36150K .......... .......... .......... .......... .......... 49% 228M 0s 2019-12-09T11:38:37.100 INFO:teuthology.orchestra.run.smithi174.stderr: 36200K .......... .......... .......... .......... .......... 49% 290M 0s 2019-12-09T11:38:37.100 INFO:teuthology.orchestra.run.smithi174.stderr: 36250K .......... .......... .......... .......... .......... 49% 45.4M 0s 2019-12-09T11:38:37.100 INFO:teuthology.orchestra.run.smithi174.stderr: 36300K .......... .......... .......... .......... .......... 49% 262M 0s 2019-12-09T11:38:37.101 INFO:teuthology.orchestra.run.smithi174.stderr: 36350K .......... .......... .......... .......... .......... 49% 221M 0s 2019-12-09T11:38:37.101 INFO:teuthology.orchestra.run.smithi174.stderr: 36400K .......... .......... .......... .......... .......... 49% 261M 0s 2019-12-09T11:38:37.101 INFO:teuthology.orchestra.run.smithi174.stderr: 36450K .......... .......... .......... .......... .......... 49% 271M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36500K .......... .......... .......... .......... .......... 49% 287M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36550K .......... .......... .......... .......... .......... 49% 251M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36600K .......... .......... .......... .......... .......... 49% 92.2M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36650K .......... .......... .......... .......... .......... 49% 262M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36700K .......... .......... .......... .......... .......... 49% 286M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36750K .......... .......... .......... .......... .......... 50% 194M 0s 2019-12-09T11:38:37.102 INFO:teuthology.orchestra.run.smithi174.stderr: 36800K .......... .......... .......... .......... .......... 50% 291M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 36850K .......... .......... .......... .......... .......... 50% 377M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 36900K .......... .......... .......... .......... .......... 50% 117M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 36950K .......... .......... .......... .......... .......... 50% 249M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 37000K .......... .......... .......... .......... .......... 50% 328M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 37050K .......... .......... .......... .......... .......... 50% 338M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 37100K .......... .......... .......... .......... .......... 50% 370M 0s 2019-12-09T11:38:37.103 INFO:teuthology.orchestra.run.smithi174.stderr: 37150K .......... .......... .......... .......... .......... 50% 299M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37200K .......... .......... .......... .......... .......... 50% 411M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37250K .......... .......... .......... .......... .......... 50% 310M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37300K .......... .......... .......... .......... .......... 50% 194M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37350K .......... .......... .......... .......... .......... 50% 219M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37400K .......... .......... .......... .......... .......... 50% 26.2M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37450K .......... .......... .......... .......... .......... 50% 250M 0s 2019-12-09T11:38:37.104 INFO:teuthology.orchestra.run.smithi174.stderr: 37500K .......... .......... .......... .......... .......... 51% 276M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37550K .......... .......... .......... .......... .......... 51% 261M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37600K .......... .......... .......... .......... .......... 51% 324M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37650K .......... .......... .......... .......... .......... 51% 322M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37700K .......... .......... .......... .......... .......... 51% 310M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37750K .......... .......... .......... .......... .......... 51% 284M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37800K .......... .......... .......... .......... .......... 51% 307M 0s 2019-12-09T11:38:37.105 INFO:teuthology.orchestra.run.smithi174.stderr: 37850K .......... .......... .......... .......... .......... 51% 294M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 37900K .......... .......... .......... .......... .......... 51% 214M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 37950K .......... .......... .......... .......... .......... 51% 170M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 38000K .......... .......... .......... .......... .......... 51% 275M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 38050K .......... .......... .......... .......... .......... 51% 333M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 38100K .......... .......... .......... .......... .......... 51% 78.9M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 38150K .......... .......... .......... .......... .......... 51% 260M 0s 2019-12-09T11:38:37.106 INFO:teuthology.orchestra.run.smithi174.stderr: 38200K .......... .......... .......... .......... .......... 51% 327M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38250K .......... .......... .......... .......... .......... 52% 345M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38300K .......... .......... .......... .......... .......... 52% 339M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38350K .......... .......... .......... .......... .......... 52% 330M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38400K .......... .......... .......... .......... .......... 52% 370M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38450K .......... .......... .......... .......... .......... 52% 391M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38500K .......... .......... .......... .......... .......... 52% 398M 0s 2019-12-09T11:38:37.107 INFO:teuthology.orchestra.run.smithi174.stderr: 38550K .......... .......... .......... .......... .......... 52% 93.7M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38600K .......... .......... .......... .......... .......... 52% 278M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38650K .......... .......... .......... .......... .......... 52% 303M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38700K .......... .......... .......... .......... .......... 52% 309M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38750K .......... .......... .......... .......... .......... 52% 279M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38800K .......... .......... .......... .......... .......... 52% 317M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38850K .......... .......... .......... .......... .......... 52% 307M 0s 2019-12-09T11:38:37.108 INFO:teuthology.orchestra.run.smithi174.stderr: 38900K .......... .......... .......... .......... .......... 52% 315M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 38950K .......... .......... .......... .......... .......... 53% 279M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39000K .......... .......... .......... .......... .......... 53% 322M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39050K .......... .......... .......... .......... .......... 53% 345M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39100K .......... .......... .......... .......... .......... 53% 400M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39150K .......... .......... .......... .......... .......... 53% 243M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39200K .......... .......... .......... .......... .......... 53% 279M 0s 2019-12-09T11:38:37.109 INFO:teuthology.orchestra.run.smithi174.stderr: 39250K .......... .......... .......... .......... .......... 53% 264M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39300K .......... .......... .......... .......... .......... 53% 129M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39350K .......... .......... .......... .......... .......... 53% 149M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39400K .......... .......... .......... .......... .......... 53% 54.1M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39450K .......... .......... .......... .......... .......... 53% 298M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39500K .......... .......... .......... .......... .......... 53% 329M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39550K .......... .......... .......... .......... .......... 53% 284M 0s 2019-12-09T11:38:37.110 INFO:teuthology.orchestra.run.smithi174.stderr: 39600K .......... .......... .......... .......... .......... 53% 369M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39650K .......... .......... .......... .......... .......... 53% 104M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39700K .......... .......... .......... .......... .......... 54% 336M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39750K .......... .......... .......... .......... .......... 54% 170M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39800K .......... .......... .......... .......... .......... 54% 272M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39850K .......... .......... .......... .......... .......... 54% 281M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39900K .......... .......... .......... .......... .......... 54% 137M 0s 2019-12-09T11:38:37.111 INFO:teuthology.orchestra.run.smithi174.stderr: 39950K .......... .......... .......... .......... .......... 54% 230M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40000K .......... .......... .......... .......... .......... 54% 289M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40050K .......... .......... .......... .......... .......... 54% 56.0M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40100K .......... .......... .......... .......... .......... 54% 166M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40150K .......... .......... .......... .......... .......... 54% 181M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40200K .......... .......... .......... .......... .......... 54% 286M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40250K .......... .......... .......... .......... .......... 54% 279M 0s 2019-12-09T11:38:37.112 INFO:teuthology.orchestra.run.smithi174.stderr: 40300K .......... .......... .......... .......... .......... 54% 291M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40350K .......... .......... .......... .......... .......... 54% 235M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40400K .......... .......... .......... .......... .......... 54% 284M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40450K .......... .......... .......... .......... .......... 55% 280M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40500K .......... .......... .......... .......... .......... 55% 273M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40550K .......... .......... .......... .......... .......... 55% 254M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40600K .......... .......... .......... .......... .......... 55% 287M 0s 2019-12-09T11:38:37.113 INFO:teuthology.orchestra.run.smithi174.stderr: 40650K .......... .......... .......... .......... .......... 55% 288M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40700K .......... .......... .......... .......... .......... 55% 267M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40750K .......... .......... .......... .......... .......... 55% 245M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40800K .......... .......... .......... .......... .......... 55% 271M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40850K .......... .......... .......... .......... .......... 55% 270M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40900K .......... .......... .......... .......... .......... 55% 277M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 40950K .......... .......... .......... .......... .......... 55% 194M 0s 2019-12-09T11:38:37.114 INFO:teuthology.orchestra.run.smithi174.stderr: 41000K .......... .......... .......... .......... .......... 55% 275M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41050K .......... .......... .......... .......... .......... 55% 88.1M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41100K .......... .......... .......... .......... .......... 55% 269M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41150K .......... .......... .......... .......... .......... 56% 257M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41200K .......... .......... .......... .......... .......... 56% 121M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41250K .......... .......... .......... .......... .......... 56% 339M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41300K .......... .......... .......... .......... .......... 56% 351M 0s 2019-12-09T11:38:37.115 INFO:teuthology.orchestra.run.smithi174.stderr: 41350K .......... .......... .......... .......... .......... 56% 201M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41400K .......... .......... .......... .......... .......... 56% 342M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41450K .......... .......... .......... .......... .......... 56% 361M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41500K .......... .......... .......... .......... .......... 56% 367M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41550K .......... .......... .......... .......... .......... 56% 343M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41600K .......... .......... .......... .......... .......... 56% 383M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41650K .......... .......... .......... .......... .......... 56% 325M 0s 2019-12-09T11:38:37.116 INFO:teuthology.orchestra.run.smithi174.stderr: 41700K .......... .......... .......... .......... .......... 56% 254M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 41750K .......... .......... .......... .......... .......... 56% 197M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 41800K .......... .......... .......... .......... .......... 56% 238M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 41850K .......... .......... .......... .......... .......... 56% 314M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 41900K .......... .......... .......... .......... .......... 57% 365M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 41950K .......... .......... .......... .......... .......... 57% 141M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 42000K .......... .......... .......... .......... .......... 57% 158M 0s 2019-12-09T11:38:37.117 INFO:teuthology.orchestra.run.smithi174.stderr: 42050K .......... .......... .......... .......... .......... 57% 132M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42100K .......... .......... .......... .......... .......... 57% 292M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42150K .......... .......... .......... .......... .......... 57% 233M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42200K .......... .......... .......... .......... .......... 57% 286M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42250K .......... .......... .......... .......... .......... 57% 277M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42300K .......... .......... .......... .......... .......... 57% 223M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42350K .......... .......... .......... .......... .......... 57% 304M 0s 2019-12-09T11:38:37.118 INFO:teuthology.orchestra.run.smithi174.stderr: 42400K .......... .......... .......... .......... .......... 57% 198M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42450K .......... .......... .......... .......... .......... 57% 124M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42500K .......... .......... .......... .......... .......... 57% 271M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42550K .......... .......... .......... .......... .......... 57% 133M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42600K .......... .......... .......... .......... .......... 57% 214M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42650K .......... .......... .......... .......... .......... 58% 182M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42700K .......... .......... .......... .......... .......... 58% 25.4M 0s 2019-12-09T11:38:37.119 INFO:teuthology.orchestra.run.smithi174.stderr: 42750K .......... .......... .......... .......... .......... 58% 231M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 42800K .......... .......... .......... .......... .......... 58% 270M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 42850K .......... .......... .......... .......... .......... 58% 280M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 42900K .......... .......... .......... .......... .......... 58% 280M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 42950K .......... .......... .......... .......... .......... 58% 260M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 43000K .......... .......... .......... .......... .......... 58% 19.8M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 43050K .......... .......... .......... .......... .......... 58% 285M 0s 2019-12-09T11:38:37.120 INFO:teuthology.orchestra.run.smithi174.stderr: 43100K .......... .......... .......... .......... .......... 58% 303M 0s 2019-12-09T11:38:37.121 INFO:teuthology.orchestra.run.smithi174.stderr: 43150K .......... .......... .......... .......... .......... 58% 279M 0s 2019-12-09T11:38:37.121 INFO:teuthology.orchestra.run.smithi174.stderr: 43200K .......... .......... .......... .......... .......... 58% 345M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43250K .......... .......... .......... .......... .......... 58% 354M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43300K .......... .......... .......... .......... .......... 58% 357M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43350K .......... .......... .......... .......... .......... 59% 321M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43400K .......... .......... .......... .......... .......... 59% 341M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43450K .......... .......... .......... .......... .......... 59% 392M 0s 2019-12-09T11:38:37.122 INFO:teuthology.orchestra.run.smithi174.stderr: 43500K .......... .......... .......... .......... .......... 59% 391M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43550K .......... .......... .......... .......... .......... 59% 339M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43600K .......... .......... .......... .......... .......... 59% 387M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43650K .......... .......... .......... .......... .......... 59% 363M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43700K .......... .......... .......... .......... .......... 59% 355M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43750K .......... .......... .......... .......... .......... 59% 67.9M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43800K .......... .......... .......... .......... .......... 59% 239M 0s 2019-12-09T11:38:37.123 INFO:teuthology.orchestra.run.smithi174.stderr: 43850K .......... .......... .......... .......... .......... 59% 246M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 43900K .......... .......... .......... .......... .......... 59% 272M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 43950K .......... .......... .......... .......... .......... 59% 260M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 44000K .......... .......... .......... .......... .......... 59% 102M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 44050K .......... .......... .......... .......... .......... 59% 251M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 44100K .......... .......... .......... .......... .......... 60% 295M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 44150K .......... .......... .......... .......... .......... 60% 248M 0s 2019-12-09T11:38:37.124 INFO:teuthology.orchestra.run.smithi174.stderr: 44200K .......... .......... .......... .......... .......... 60% 297M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44250K .......... .......... .......... .......... .......... 60% 298M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44300K .......... .......... .......... .......... .......... 60% 284M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44350K .......... .......... .......... .......... .......... 60% 25.6M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44400K .......... .......... .......... .......... .......... 60% 322M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44450K .......... .......... .......... .......... .......... 60% 383M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44500K .......... .......... .......... .......... .......... 60% 406M 0s 2019-12-09T11:38:37.125 INFO:teuthology.orchestra.run.smithi174.stderr: 44550K .......... .......... .......... .......... .......... 60% 166M 0s 2019-12-09T11:38:37.126 INFO:teuthology.orchestra.run.smithi174.stderr: 44600K .......... .......... .......... .......... .......... 60% 234M 0s 2019-12-09T11:38:37.126 INFO:teuthology.orchestra.run.smithi174.stderr: 44650K .......... .......... .......... .......... .......... 60% 301M 0s 2019-12-09T11:38:37.126 INFO:teuthology.orchestra.run.smithi174.stderr: 44700K .......... .......... .......... .......... .......... 60% 286M 0s 2019-12-09T11:38:37.126 INFO:teuthology.orchestra.run.smithi174.stderr: 44750K .......... .......... .......... .......... .......... 60% 64.9M 0s 2019-12-09T11:38:37.127 INFO:teuthology.orchestra.run.smithi174.stderr: 44800K .......... .......... .......... .......... .......... 60% 199M 0s 2019-12-09T11:38:37.127 INFO:teuthology.orchestra.run.smithi174.stderr: 44850K .......... .......... .......... .......... .......... 61% 231M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 44900K .......... .......... .......... .......... .......... 61% 293M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 44950K .......... .......... .......... .......... .......... 61% 261M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 45000K .......... .......... .......... .......... .......... 61% 299M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 45050K .......... .......... .......... .......... .......... 61% 261M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 45100K .......... .......... .......... .......... .......... 61% 289M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 45150K .......... .......... .......... .......... .......... 61% 271M 0s 2019-12-09T11:38:37.128 INFO:teuthology.orchestra.run.smithi174.stderr: 45200K .......... .......... .......... .......... .......... 61% 61.8M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45250K .......... .......... .......... .......... .......... 61% 300M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45300K .......... .......... .......... .......... .......... 61% 332M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45350K .......... .......... .......... .......... .......... 61% 284M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45400K .......... .......... .......... .......... .......... 61% 228M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45450K .......... .......... .......... .......... .......... 61% 232M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45500K .......... .......... .......... .......... .......... 61% 54.2M 0s 2019-12-09T11:38:37.129 INFO:teuthology.orchestra.run.smithi174.stderr: 45550K .......... .......... .......... .......... .......... 61% 195M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45600K .......... .......... .......... .......... .......... 62% 224M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45650K .......... .......... .......... .......... .......... 62% 228M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45700K .......... .......... .......... .......... .......... 62% 229M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45750K .......... .......... .......... .......... .......... 62% 233M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45800K .......... .......... .......... .......... .......... 62% 287M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45850K .......... .......... .......... .......... .......... 62% 295M 0s 2019-12-09T11:38:37.130 INFO:teuthology.orchestra.run.smithi174.stderr: 45900K .......... .......... .......... .......... .......... 62% 273M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 45950K .......... .......... .......... .......... .......... 62% 243M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46000K .......... .......... .......... .......... .......... 62% 284M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46050K .......... .......... .......... .......... .......... 62% 275M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46100K .......... .......... .......... .......... .......... 62% 297M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46150K .......... .......... .......... .......... .......... 62% 153M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46200K .......... .......... .......... .......... .......... 62% 359M 0s 2019-12-09T11:38:37.131 INFO:teuthology.orchestra.run.smithi174.stderr: 46250K .......... .......... .......... .......... .......... 62% 87.9M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46300K .......... .......... .......... .......... .......... 63% 335M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46350K .......... .......... .......... .......... .......... 63% 58.9M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46400K .......... .......... .......... .......... .......... 63% 252M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46450K .......... .......... .......... .......... .......... 63% 340M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46500K .......... .......... .......... .......... .......... 63% 345M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46550K .......... .......... .......... .......... .......... 63% 141M 0s 2019-12-09T11:38:37.132 INFO:teuthology.orchestra.run.smithi174.stderr: 46600K .......... .......... .......... .......... .......... 63% 186M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46650K .......... .......... .......... .......... .......... 63% 240M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46700K .......... .......... .......... .......... .......... 63% 289M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46750K .......... .......... .......... .......... .......... 63% 239M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46800K .......... .......... .......... .......... .......... 63% 265M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46850K .......... .......... .......... .......... .......... 63% 289M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46900K .......... .......... .......... .......... .......... 63% 259M 0s 2019-12-09T11:38:37.133 INFO:teuthology.orchestra.run.smithi174.stderr: 46950K .......... .......... .......... .......... .......... 63% 102M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47000K .......... .......... .......... .......... .......... 63% 248M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47050K .......... .......... .......... .......... .......... 64% 390M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47100K .......... .......... .......... .......... .......... 64% 115M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47150K .......... .......... .......... .......... .......... 64% 315M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47200K .......... .......... .......... .......... .......... 64% 403M 0s 2019-12-09T11:38:37.134 INFO:teuthology.orchestra.run.smithi174.stderr: 47250K .......... .......... .......... .......... .......... 64% 371M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47300K .......... .......... .......... .......... .......... 64% 414M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47350K .......... .......... .......... .......... .......... 64% 75.7M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47400K .......... .......... .......... .......... .......... 64% 266M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47450K .......... .......... .......... .......... .......... 64% 293M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47500K .......... .......... .......... .......... .......... 64% 46.7M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47550K .......... .......... .......... .......... .......... 64% 261M 0s 2019-12-09T11:38:37.135 INFO:teuthology.orchestra.run.smithi174.stderr: 47600K .......... .......... .......... .......... .......... 64% 344M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47650K .......... .......... .......... .......... .......... 64% 378M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47700K .......... .......... .......... .......... .......... 64% 324M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47750K .......... .......... .......... .......... .......... 64% 313M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47800K .......... .......... .......... .......... .......... 65% 429M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47850K .......... .......... .......... .......... .......... 65% 176M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47900K .......... .......... .......... .......... .......... 65% 335M 0s 2019-12-09T11:38:37.136 INFO:teuthology.orchestra.run.smithi174.stderr: 47950K .......... .......... .......... .......... .......... 65% 71.8M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48000K .......... .......... .......... .......... .......... 65% 333M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48050K .......... .......... .......... .......... .......... 65% 359M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48100K .......... .......... .......... .......... .......... 65% 288M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48150K .......... .......... .......... .......... .......... 65% 89.0M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48200K .......... .......... .......... .......... .......... 65% 193M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48250K .......... .......... .......... .......... .......... 65% 218M 0s 2019-12-09T11:38:37.137 INFO:teuthology.orchestra.run.smithi174.stderr: 48300K .......... .......... .......... .......... .......... 65% 255M 0s 2019-12-09T11:38:37.138 INFO:teuthology.orchestra.run.smithi174.stderr: 48350K .......... .......... .......... .......... .......... 65% 218M 0s 2019-12-09T11:38:37.138 INFO:teuthology.orchestra.run.smithi174.stderr: 48400K .......... .......... .......... .......... .......... 65% 298M 0s 2019-12-09T11:38:37.138 INFO:teuthology.orchestra.run.smithi174.stderr: 48450K .......... .......... .......... .......... .......... 65% 69.2M 0s 2019-12-09T11:38:37.138 INFO:teuthology.orchestra.run.smithi174.stderr: 48500K .......... .......... .......... .......... .......... 66% 312M 0s 2019-12-09T11:38:37.138 INFO:teuthology.orchestra.run.smithi174.stderr: 48550K .......... .......... .......... .......... .......... 66% 274M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48600K .......... .......... .......... .......... .......... 66% 362M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48650K .......... .......... .......... .......... .......... 66% 384M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48700K .......... .......... .......... .......... .......... 66% 148M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48750K .......... .......... .......... .......... .......... 66% 276M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48800K .......... .......... .......... .......... .......... 66% 344M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48850K .......... .......... .......... .......... .......... 66% 209M 0s 2019-12-09T11:38:37.139 INFO:teuthology.orchestra.run.smithi174.stderr: 48900K .......... .......... .......... .......... .......... 66% 223M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 48950K .......... .......... .......... .......... .......... 66% 209M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 49000K .......... .......... .......... .......... .......... 66% 223M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 49050K .......... .......... .......... .......... .......... 66% 293M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 49100K .......... .......... .......... .......... .......... 66% 295M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 49150K .......... .......... .......... .......... .......... 66% 180M 0s 2019-12-09T11:38:37.140 INFO:teuthology.orchestra.run.smithi174.stderr: 49200K .......... .......... .......... .......... .......... 66% 293M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49250K .......... .......... .......... .......... .......... 67% 235M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49300K .......... .......... .......... .......... .......... 67% 217M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49350K .......... .......... .......... .......... .......... 67% 238M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49400K .......... .......... .......... .......... .......... 67% 124M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49450K .......... .......... .......... .......... .......... 67% 264M 0s 2019-12-09T11:38:37.141 INFO:teuthology.orchestra.run.smithi174.stderr: 49500K .......... .......... .......... .......... .......... 67% 218M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49550K .......... .......... .......... .......... .......... 67% 250M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49600K .......... .......... .......... .......... .......... 67% 378M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49650K .......... .......... .......... .......... .......... 67% 23.8M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49700K .......... .......... .......... .......... .......... 67% 326M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49750K .......... .......... .......... .......... .......... 67% 300M 0s 2019-12-09T11:38:37.142 INFO:teuthology.orchestra.run.smithi174.stderr: 49800K .......... .......... .......... .......... .......... 67% 342M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 49850K .......... .......... .......... .......... .......... 67% 369M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 49900K .......... .......... .......... .......... .......... 67% 333M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 49950K .......... .......... .......... .......... .......... 67% 342M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 50000K .......... .......... .......... .......... .......... 68% 413M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 50050K .......... .......... .......... .......... .......... 68% 405M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 50100K .......... .......... .......... .......... .......... 68% 399M 0s 2019-12-09T11:38:37.143 INFO:teuthology.orchestra.run.smithi174.stderr: 50150K .......... .......... .......... .......... .......... 68% 319M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50200K .......... .......... .......... .......... .......... 68% 245M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50250K .......... .......... .......... .......... .......... 68% 272M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50300K .......... .......... .......... .......... .......... 68% 195M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50350K .......... .......... .......... .......... .......... 68% 227M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50400K .......... .......... .......... .......... .......... 68% 42.4M 0s 2019-12-09T11:38:37.144 INFO:teuthology.orchestra.run.smithi174.stderr: 50450K .......... .......... .......... .......... .......... 68% 304M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50500K .......... .......... .......... .......... .......... 68% 359M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50550K .......... .......... .......... .......... .......... 68% 362M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50600K .......... .......... .......... .......... .......... 68% 410M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50650K .......... .......... .......... .......... .......... 68% 416M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50700K .......... .......... .......... .......... .......... 68% 401M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50750K .......... .......... .......... .......... .......... 69% 197M 0s 2019-12-09T11:38:37.145 INFO:teuthology.orchestra.run.smithi174.stderr: 50800K .......... .......... .......... .......... .......... 69% 259M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 50850K .......... .......... .......... .......... .......... 69% 271M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 50900K .......... .......... .......... .......... .......... 69% 133M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 50950K .......... .......... .......... .......... .......... 69% 198M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 51000K .......... .......... .......... .......... .......... 69% 405M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 51050K .......... .......... .......... .......... .......... 69% 177M 0s 2019-12-09T11:38:37.146 INFO:teuthology.orchestra.run.smithi174.stderr: 51100K .......... .......... .......... .......... .......... 69% 228M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51150K .......... .......... .......... .......... .......... 69% 226M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51200K .......... .......... .......... .......... .......... 69% 229M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51250K .......... .......... .......... .......... .......... 69% 308M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51300K .......... .......... .......... .......... .......... 69% 115M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51350K .......... .......... .......... .......... .......... 69% 239M 0s 2019-12-09T11:38:37.147 INFO:teuthology.orchestra.run.smithi174.stderr: 51400K .......... .......... .......... .......... .......... 69% 286M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51450K .......... .......... .......... .......... .......... 70% 122M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51500K .......... .......... .......... .......... .......... 70% 263M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51550K .......... .......... .......... .......... .......... 70% 222M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51600K .......... .......... .......... .......... .......... 70% 298M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51650K .......... .......... .......... .......... .......... 70% 108M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51700K .......... .......... .......... .......... .......... 70% 203M 0s 2019-12-09T11:38:37.148 INFO:teuthology.orchestra.run.smithi174.stderr: 51750K .......... .......... .......... .......... .......... 70% 223M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 51800K .......... .......... .......... .......... .......... 70% 99.3M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 51850K .......... .......... .......... .......... .......... 70% 292M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 51900K .......... .......... .......... .......... .......... 70% 143M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 51950K .......... .......... .......... .......... .......... 70% 305M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 52000K .......... .......... .......... .......... .......... 70% 48.5M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 52050K .......... .......... .......... .......... .......... 70% 363M 0s 2019-12-09T11:38:37.149 INFO:teuthology.orchestra.run.smithi174.stderr: 52100K .......... .......... .......... .......... .......... 70% 318M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52150K .......... .......... .......... .......... .......... 70% 303M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52200K .......... .......... .......... .......... .......... 71% 406M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52250K .......... .......... .......... .......... .......... 71% 92.6M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52300K .......... .......... .......... .......... .......... 71% 261M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52350K .......... .......... .......... .......... .......... 71% 210M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52400K .......... .......... .......... .......... .......... 71% 229M 0s 2019-12-09T11:38:37.150 INFO:teuthology.orchestra.run.smithi174.stderr: 52450K .......... .......... .......... .......... .......... 71% 59.5M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52500K .......... .......... .......... .......... .......... 71% 244M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52550K .......... .......... .......... .......... .......... 71% 274M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52600K .......... .......... .......... .......... .......... 71% 106M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52650K .......... .......... .......... .......... .......... 71% 186M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52700K .......... .......... .......... .......... .......... 71% 228M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52750K .......... .......... .......... .......... .......... 71% 180M 0s 2019-12-09T11:38:37.151 INFO:teuthology.orchestra.run.smithi174.stderr: 52800K .......... .......... .......... .......... .......... 71% 228M 0s 2019-12-09T11:38:37.152 INFO:teuthology.orchestra.run.smithi174.stderr: 52850K .......... .......... .......... .......... .......... 71% 276M 0s 2019-12-09T11:38:37.152 INFO:teuthology.orchestra.run.smithi174.stderr: 52900K .......... .......... .......... .......... .......... 71% 230M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 52950K .......... .......... .......... .......... .......... 72% 248M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 53000K .......... .......... .......... .......... .......... 72% 295M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 53050K .......... .......... .......... .......... .......... 72% 235M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 53100K .......... .......... .......... .......... .......... 72% 201M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 53150K .......... .......... .......... .......... .......... 72% 289M 0s 2019-12-09T11:38:37.153 INFO:teuthology.orchestra.run.smithi174.stderr: 53200K .......... .......... .......... .......... .......... 72% 135M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53250K .......... .......... .......... .......... .......... 72% 253M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53300K .......... .......... .......... .......... .......... 72% 308M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53350K .......... .......... .......... .......... .......... 72% 51.0M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53400K .......... .......... .......... .......... .......... 72% 381M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53450K .......... .......... .......... .......... .......... 72% 362M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53500K .......... .......... .......... .......... .......... 72% 359M 0s 2019-12-09T11:38:37.154 INFO:teuthology.orchestra.run.smithi174.stderr: 53550K .......... .......... .......... .......... .......... 72% 320M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53600K .......... .......... .......... .......... .......... 72% 404M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53650K .......... .......... .......... .......... .......... 73% 266M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53700K .......... .......... .......... .......... .......... 73% 335M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53750K .......... .......... .......... .......... .......... 73% 250M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53800K .......... .......... .......... .......... .......... 73% 346M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53850K .......... .......... .......... .......... .......... 73% 357M 0s 2019-12-09T11:38:37.155 INFO:teuthology.orchestra.run.smithi174.stderr: 53900K .......... .......... .......... .......... .......... 73% 150M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 53950K .......... .......... .......... .......... .......... 73% 297M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54000K .......... .......... .......... .......... .......... 73% 81.5M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54050K .......... .......... .......... .......... .......... 73% 241M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54100K .......... .......... .......... .......... .......... 73% 273M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54150K .......... .......... .......... .......... .......... 73% 247M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54200K .......... .......... .......... .......... .......... 73% 299M 0s 2019-12-09T11:38:37.156 INFO:teuthology.orchestra.run.smithi174.stderr: 54250K .......... .......... .......... .......... .......... 73% 84.8M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54300K .......... .......... .......... .......... .......... 73% 281M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54350K .......... .......... .......... .......... .......... 73% 86.2M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54400K .......... .......... .......... .......... .......... 74% 277M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54450K .......... .......... .......... .......... .......... 74% 341M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54500K .......... .......... .......... .......... .......... 74% 256M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54550K .......... .......... .......... .......... .......... 74% 105M 0s 2019-12-09T11:38:37.157 INFO:teuthology.orchestra.run.smithi174.stderr: 54600K .......... .......... .......... .......... .......... 74% 215M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54650K .......... .......... .......... .......... .......... 74% 230M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54700K .......... .......... .......... .......... .......... 74% 249M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54750K .......... .......... .......... .......... .......... 74% 241M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54800K .......... .......... .......... .......... .......... 74% 160M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54850K .......... .......... .......... .......... .......... 74% 256M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54900K .......... .......... .......... .......... .......... 74% 270M 0s 2019-12-09T11:38:37.158 INFO:teuthology.orchestra.run.smithi174.stderr: 54950K .......... .......... .......... .......... .......... 74% 259M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55000K .......... .......... .......... .......... .......... 74% 76.4M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55050K .......... .......... .......... .......... .......... 74% 387M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55100K .......... .......... .......... .......... .......... 74% 391M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55150K .......... .......... .......... .......... .......... 75% 184M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55200K .......... .......... .......... .......... .......... 75% 241M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55250K .......... .......... .......... .......... .......... 75% 83.1M 0s 2019-12-09T11:38:37.159 INFO:teuthology.orchestra.run.smithi174.stderr: 55300K .......... .......... .......... .......... .......... 75% 235M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55350K .......... .......... .......... .......... .......... 75% 127M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55400K .......... .......... .......... .......... .......... 75% 234M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55450K .......... .......... .......... .......... .......... 75% 271M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55500K .......... .......... .......... .......... .......... 75% 297M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55550K .......... .......... .......... .......... .......... 75% 251M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55600K .......... .......... .......... .......... .......... 75% 295M 0s 2019-12-09T11:38:37.160 INFO:teuthology.orchestra.run.smithi174.stderr: 55650K .......... .......... .......... .......... .......... 75% 204M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55700K .......... .......... .......... .......... .......... 75% 284M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55750K .......... .......... .......... .......... .......... 75% 244M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55800K .......... .......... .......... .......... .......... 75% 24.7M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55850K .......... .......... .......... .......... .......... 75% 279M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55900K .......... .......... .......... .......... .......... 76% 285M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 55950K .......... .......... .......... .......... .......... 76% 244M 0s 2019-12-09T11:38:37.161 INFO:teuthology.orchestra.run.smithi174.stderr: 56000K .......... .......... .......... .......... .......... 76% 303M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56050K .......... .......... .......... .......... .......... 76% 52.1M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56100K .......... .......... .......... .......... .......... 76% 298M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56150K .......... .......... .......... .......... .......... 76% 92.6M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56200K .......... .......... .......... .......... .......... 76% 234M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56250K .......... .......... .......... .......... .......... 76% 286M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56300K .......... .......... .......... .......... .......... 76% 40.1M 0s 2019-12-09T11:38:37.162 INFO:teuthology.orchestra.run.smithi174.stderr: 56350K .......... .......... .......... .......... .......... 76% 184M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56400K .......... .......... .......... .......... .......... 76% 224M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56450K .......... .......... .......... .......... .......... 76% 281M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56500K .......... .......... .......... .......... .......... 76% 283M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56550K .......... .......... .......... .......... .......... 76% 245M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56600K .......... .......... .......... .......... .......... 77% 269M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56650K .......... .......... .......... .......... .......... 77% 278M 0s 2019-12-09T11:38:37.163 INFO:teuthology.orchestra.run.smithi174.stderr: 56700K .......... .......... .......... .......... .......... 77% 272M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 56750K .......... .......... .......... .......... .......... 77% 224M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 56800K .......... .......... .......... .......... .......... 77% 272M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 56850K .......... .......... .......... .......... .......... 77% 269M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 56900K .......... .......... .......... .......... .......... 77% 282M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 56950K .......... .......... .......... .......... .......... 77% 252M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 57000K .......... .......... .......... .......... .......... 77% 279M 0s 2019-12-09T11:38:37.164 INFO:teuthology.orchestra.run.smithi174.stderr: 57050K .......... .......... .......... .......... .......... 77% 282M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57100K .......... .......... .......... .......... .......... 77% 296M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57150K .......... .......... .......... .......... .......... 77% 230M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57200K .......... .......... .......... .......... .......... 77% 283M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57250K .......... .......... .......... .......... .......... 77% 286M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57300K .......... .......... .......... .......... .......... 77% 272M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57350K .......... .......... .......... .......... .......... 78% 240M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57400K .......... .......... .......... .......... .......... 78% 276M 0s 2019-12-09T11:38:37.165 INFO:teuthology.orchestra.run.smithi174.stderr: 57450K .......... .......... .......... .......... .......... 78% 277M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57500K .......... .......... .......... .......... .......... 78% 291M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57550K .......... .......... .......... .......... .......... 78% 190M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57600K .......... .......... .......... .......... .......... 78% 262M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57650K .......... .......... .......... .......... .......... 78% 328M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57700K .......... .......... .......... .......... .......... 78% 305M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57750K .......... .......... .......... .......... .......... 78% 53.2M 0s 2019-12-09T11:38:37.166 INFO:teuthology.orchestra.run.smithi174.stderr: 57800K .......... .......... .......... .......... .......... 78% 290M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 57850K .......... .......... .......... .......... .......... 78% 342M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 57900K .......... .......... .......... .......... .......... 78% 395M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 57950K .......... .......... .......... .......... .......... 78% 346M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 58000K .......... .......... .......... .......... .......... 78% 297M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 58050K .......... .......... .......... .......... .......... 78% 253M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 58100K .......... .......... .......... .......... .......... 79% 43.9M 0s 2019-12-09T11:38:37.167 INFO:teuthology.orchestra.run.smithi174.stderr: 58150K .......... .......... .......... .......... .......... 79% 243M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58200K .......... .......... .......... .......... .......... 79% 297M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58250K .......... .......... .......... .......... .......... 79% 287M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58300K .......... .......... .......... .......... .......... 79% 279M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58350K .......... .......... .......... .......... .......... 79% 245M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58400K .......... .......... .......... .......... .......... 79% 280M 0s 2019-12-09T11:38:37.168 INFO:teuthology.orchestra.run.smithi174.stderr: 58450K .......... .......... .......... .......... .......... 79% 294M 0s 2019-12-09T11:38:37.169 INFO:teuthology.orchestra.run.smithi174.stderr: 58500K .......... .......... .......... .......... .......... 79% 287M 0s 2019-12-09T11:38:37.169 INFO:teuthology.orchestra.run.smithi174.stderr: 58550K .......... .......... .......... .......... .......... 79% 259M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58600K .......... .......... .......... .......... .......... 79% 295M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58650K .......... .......... .......... .......... .......... 79% 288M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58700K .......... .......... .......... .......... .......... 79% 297M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58750K .......... .......... .......... .......... .......... 79% 252M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58800K .......... .......... .......... .......... .......... 80% 303M 0s 2019-12-09T11:38:37.170 INFO:teuthology.orchestra.run.smithi174.stderr: 58850K .......... .......... .......... .......... .......... 80% 79.1M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 58900K .......... .......... .......... .......... .......... 80% 284M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 58950K .......... .......... .......... .......... .......... 80% 244M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 59000K .......... .......... .......... .......... .......... 80% 276M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 59050K .......... .......... .......... .......... .......... 80% 202M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 59100K .......... .......... .......... .......... .......... 80% 219M 0s 2019-12-09T11:38:37.171 INFO:teuthology.orchestra.run.smithi174.stderr: 59150K .......... .......... .......... .......... .......... 80% 79.7M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59200K .......... .......... .......... .......... .......... 80% 281M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59250K .......... .......... .......... .......... .......... 80% 79.0M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59300K .......... .......... .......... .......... .......... 80% 291M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59350K .......... .......... .......... .......... .......... 80% 138M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59400K .......... .......... .......... .......... .......... 80% 284M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59450K .......... .......... .......... .......... .......... 80% 290M 0s 2019-12-09T11:38:37.172 INFO:teuthology.orchestra.run.smithi174.stderr: 59500K .......... .......... .......... .......... .......... 80% 104M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59550K .......... .......... .......... .......... .......... 81% 222M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59600K .......... .......... .......... .......... .......... 81% 294M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59650K .......... .......... .......... .......... .......... 81% 360M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59700K .......... .......... .......... .......... .......... 81% 353M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59750K .......... .......... .......... .......... .......... 81% 179M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59800K .......... .......... .......... .......... .......... 81% 319M 0s 2019-12-09T11:38:37.173 INFO:teuthology.orchestra.run.smithi174.stderr: 59850K .......... .......... .......... .......... .......... 81% 387M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 59900K .......... .......... .......... .......... .......... 81% 116M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 59950K .......... .......... .......... .......... .......... 81% 233M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 60000K .......... .......... .......... .......... .......... 81% 92.9M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 60050K .......... .......... .......... .......... .......... 81% 278M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 60100K .......... .......... .......... .......... .......... 81% 295M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 60150K .......... .......... .......... .......... .......... 81% 237M 0s 2019-12-09T11:38:37.174 INFO:teuthology.orchestra.run.smithi174.stderr: 60200K .......... .......... .......... .......... .......... 81% 285M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60250K .......... .......... .......... .......... .......... 81% 111M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60300K .......... .......... .......... .......... .......... 82% 288M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60350K .......... .......... .......... .......... .......... 82% 269M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60400K .......... .......... .......... .......... .......... 82% 301M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60450K .......... .......... .......... .......... .......... 82% 302M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60500K .......... .......... .......... .......... .......... 82% 300M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60550K .......... .......... .......... .......... .......... 82% 37.0M 0s 2019-12-09T11:38:37.175 INFO:teuthology.orchestra.run.smithi174.stderr: 60600K .......... .......... .......... .......... .......... 82% 283M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60650K .......... .......... .......... .......... .......... 82% 283M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60700K .......... .......... .......... .......... .......... 82% 304M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60750K .......... .......... .......... .......... .......... 82% 315M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60800K .......... .......... .......... .......... .......... 82% 351M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60850K .......... .......... .......... .......... .......... 82% 336M 0s 2019-12-09T11:38:37.176 INFO:teuthology.orchestra.run.smithi174.stderr: 60900K .......... .......... .......... .......... .......... 82% 91.4M 0s 2019-12-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi174.stderr: 60950K .......... .......... .......... .......... .......... 82% 323M 0s 2019-12-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi174.stderr: 61000K .......... .......... .......... .......... .......... 82% 400M 0s 2019-12-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi174.stderr: 61050K .......... .......... .......... .......... .......... 83% 411M 0s 2019-12-09T11:38:37.177 INFO:teuthology.orchestra.run.smithi174.stderr: 61100K .......... .......... .......... .......... .......... 83% 390M 0s 2019-12-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi174.stderr: 61150K .......... .......... .......... .......... .......... 83% 194M 0s 2019-12-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi174.stderr: 61200K .......... .......... .......... .......... .......... 83% 396M 0s 2019-12-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi174.stderr: 61250K .......... .......... .......... .......... .......... 83% 324M 0s 2019-12-09T11:38:37.178 INFO:teuthology.orchestra.run.smithi174.stderr: 61300K .......... .......... .......... .......... .......... 83% 225M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61350K .......... .......... .......... .......... .......... 83% 372M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61400K .......... .......... .......... .......... .......... 83% 35.8M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61450K .......... .......... .......... .......... .......... 83% 219M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61500K .......... .......... .......... .......... .......... 83% 287M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61550K .......... .......... .......... .......... .......... 83% 240M 0s 2019-12-09T11:38:37.179 INFO:teuthology.orchestra.run.smithi174.stderr: 61600K .......... .......... .......... .......... .......... 83% 293M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61650K .......... .......... .......... .......... .......... 83% 295M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61700K .......... .......... .......... .......... .......... 83% 281M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61750K .......... .......... .......... .......... .......... 84% 256M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61800K .......... .......... .......... .......... .......... 84% 298M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61850K .......... .......... .......... .......... .......... 84% 294M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61900K .......... .......... .......... .......... .......... 84% 291M 0s 2019-12-09T11:38:37.180 INFO:teuthology.orchestra.run.smithi174.stderr: 61950K .......... .......... .......... .......... .......... 84% 240M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62000K .......... .......... .......... .......... .......... 84% 300M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62050K .......... .......... .......... .......... .......... 84% 44.2M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62100K .......... .......... .......... .......... .......... 84% 217M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62150K .......... .......... .......... .......... .......... 84% 213M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62200K .......... .......... .......... .......... .......... 84% 282M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62250K .......... .......... .......... .......... .......... 84% 274M 0s 2019-12-09T11:38:37.181 INFO:teuthology.orchestra.run.smithi174.stderr: 62300K .......... .......... .......... .......... .......... 84% 265M 0s 2019-12-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi174.stderr: 62350K .......... .......... .......... .......... .......... 84% 245M 0s 2019-12-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi174.stderr: 62400K .......... .......... .......... .......... .......... 84% 276M 0s 2019-12-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi174.stderr: 62450K .......... .......... .......... .......... .......... 84% 273M 0s 2019-12-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi174.stderr: 62500K .......... .......... .......... .......... .......... 85% 281M 0s 2019-12-09T11:38:37.182 INFO:teuthology.orchestra.run.smithi174.stderr: 62550K .......... .......... .......... .......... .......... 85% 230M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62600K .......... .......... .......... .......... .......... 85% 279M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62650K .......... .......... .......... .......... .......... 85% 291M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62700K .......... .......... .......... .......... .......... 85% 295M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62750K .......... .......... .......... .......... .......... 85% 234M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62800K .......... .......... .......... .......... .......... 85% 288M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62850K .......... .......... .......... .......... .......... 85% 273M 0s 2019-12-09T11:38:37.183 INFO:teuthology.orchestra.run.smithi174.stderr: 62900K .......... .......... .......... .......... .......... 85% 292M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 62950K .......... .......... .......... .......... .......... 85% 143M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63000K .......... .......... .......... .......... .......... 85% 290M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63050K .......... .......... .......... .......... .......... 85% 196M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63100K .......... .......... .......... .......... .......... 85% 308M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63150K .......... .......... .......... .......... .......... 85% 256M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63200K .......... .......... .......... .......... .......... 85% 57.9M 0s 2019-12-09T11:38:37.184 INFO:teuthology.orchestra.run.smithi174.stderr: 63250K .......... .......... .......... .......... .......... 86% 299M 0s 2019-12-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr: 63300K .......... .......... .......... .......... .......... 86% 342M 0s 2019-12-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr: 63350K .......... .......... .......... .......... .......... 86% 292M 0s 2019-12-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr: 63400K .......... .......... .......... .......... .......... 86% 356M 0s 2019-12-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr: 63450K .......... .......... .......... .......... .......... 86% 28.7M 0s 2019-12-09T11:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr: 63500K .......... .......... .......... .......... .......... 86% 40.6M 0s 2019-12-09T11:38:37.186 INFO:teuthology.orchestra.run.smithi174.stderr: 63550K .......... .......... .......... .......... .......... 86% 251M 0s 2019-12-09T11:38:37.186 INFO:teuthology.orchestra.run.smithi174.stderr: 63600K .......... .......... .......... .......... .......... 86% 282M 0s 2019-12-09T11:38:37.187 INFO:teuthology.orchestra.run.smithi174.stderr: 63650K .......... .......... .......... .......... .......... 86% 282M 0s 2019-12-09T11:38:37.187 INFO:teuthology.orchestra.run.smithi174.stderr: 63700K .......... .......... .......... .......... .......... 86% 293M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 63750K .......... .......... .......... .......... .......... 86% 227M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 63800K .......... .......... .......... .......... .......... 86% 300M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 63850K .......... .......... .......... .......... .......... 86% 297M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 63900K .......... .......... .......... .......... .......... 86% 288M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 63950K .......... .......... .......... .......... .......... 87% 268M 0s 2019-12-09T11:38:37.188 INFO:teuthology.orchestra.run.smithi174.stderr: 64000K .......... .......... .......... .......... .......... 87% 292M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64050K .......... .......... .......... .......... .......... 87% 295M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64100K .......... .......... .......... .......... .......... 87% 286M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64150K .......... .......... .......... .......... .......... 87% 246M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64200K .......... .......... .......... .......... .......... 87% 28.6M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64250K .......... .......... .......... .......... .......... 87% 283M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64300K .......... .......... .......... .......... .......... 87% 285M 0s 2019-12-09T11:38:37.189 INFO:teuthology.orchestra.run.smithi174.stderr: 64350K .......... .......... .......... .......... .......... 87% 92.4M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64400K .......... .......... .......... .......... .......... 87% 284M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64450K .......... .......... .......... .......... .......... 87% 296M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64500K .......... .......... .......... .......... .......... 87% 49.0M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64550K .......... .......... .......... .......... .......... 87% 256M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64600K .......... .......... .......... .......... .......... 87% 285M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64650K .......... .......... .......... .......... .......... 87% 295M 0s 2019-12-09T11:38:37.190 INFO:teuthology.orchestra.run.smithi174.stderr: 64700K .......... .......... .......... .......... .......... 88% 299M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 64750K .......... .......... .......... .......... .......... 88% 154M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 64800K .......... .......... .......... .......... .......... 88% 283M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 64850K .......... .......... .......... .......... .......... 88% 263M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 64900K .......... .......... .......... .......... .......... 88% 288M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 64950K .......... .......... .......... .......... .......... 88% 256M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 65000K .......... .......... .......... .......... .......... 88% 124M 0s 2019-12-09T11:38:37.191 INFO:teuthology.orchestra.run.smithi174.stderr: 65050K .......... .......... .......... .......... .......... 88% 229M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65100K .......... .......... .......... .......... .......... 88% 125M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65150K .......... .......... .......... .......... .......... 88% 189M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65200K .......... .......... .......... .......... .......... 88% 282M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65250K .......... .......... .......... .......... .......... 88% 87.0M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65300K .......... .......... .......... .......... .......... 88% 315M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65350K .......... .......... .......... .......... .......... 88% 163M 0s 2019-12-09T11:38:37.192 INFO:teuthology.orchestra.run.smithi174.stderr: 65400K .......... .......... .......... .......... .......... 88% 257M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65450K .......... .......... .......... .......... .......... 89% 320M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65500K .......... .......... .......... .......... .......... 89% 269M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65550K .......... .......... .......... .......... .......... 89% 218M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65600K .......... .......... .......... .......... .......... 89% 419M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65650K .......... .......... .......... .......... .......... 89% 89.3M 0s 2019-12-09T11:38:37.193 INFO:teuthology.orchestra.run.smithi174.stderr: 65700K .......... .......... .......... .......... .......... 89% 414M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 65750K .......... .......... .......... .......... .......... 89% 69.3M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 65800K .......... .......... .......... .......... .......... 89% 311M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 65850K .......... .......... .......... .......... .......... 89% 361M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 65900K .......... .......... .......... .......... .......... 89% 127M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 65950K .......... .......... .......... .......... .......... 89% 298M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 66000K .......... .......... .......... .......... .......... 89% 107M 0s 2019-12-09T11:38:37.194 INFO:teuthology.orchestra.run.smithi174.stderr: 66050K .......... .......... .......... .......... .......... 89% 404M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66100K .......... .......... .......... .......... .......... 89% 65.1M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66150K .......... .......... .......... .......... .......... 89% 308M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66200K .......... .......... .......... .......... .......... 90% 216M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66250K .......... .......... .......... .......... .......... 90% 299M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66300K .......... .......... .......... .......... .......... 90% 103M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66350K .......... .......... .......... .......... .......... 90% 230M 0s 2019-12-09T11:38:37.195 INFO:teuthology.orchestra.run.smithi174.stderr: 66400K .......... .......... .......... .......... .......... 90% 224M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66450K .......... .......... .......... .......... .......... 90% 244M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66500K .......... .......... .......... .......... .......... 90% 291M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66550K .......... .......... .......... .......... .......... 90% 252M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66600K .......... .......... .......... .......... .......... 90% 40.1M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66650K .......... .......... .......... .......... .......... 90% 290M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66700K .......... .......... .......... .......... .......... 90% 283M 0s 2019-12-09T11:38:37.196 INFO:teuthology.orchestra.run.smithi174.stderr: 66750K .......... .......... .......... .......... .......... 90% 241M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 66800K .......... .......... .......... .......... .......... 90% 299M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 66850K .......... .......... .......... .......... .......... 90% 110M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 66900K .......... .......... .......... .......... .......... 91% 205M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 66950K .......... .......... .......... .......... .......... 91% 189M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 67000K .......... .......... .......... .......... .......... 91% 292M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 67050K .......... .......... .......... .......... .......... 91% 98.5M 0s 2019-12-09T11:38:37.197 INFO:teuthology.orchestra.run.smithi174.stderr: 67100K .......... .......... .......... .......... .......... 91% 256M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67150K .......... .......... .......... .......... .......... 91% 217M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67200K .......... .......... .......... .......... .......... 91% 285M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67250K .......... .......... .......... .......... .......... 91% 286M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67300K .......... .......... .......... .......... .......... 91% 64.6M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67350K .......... .......... .......... .......... .......... 91% 249M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67400K .......... .......... .......... .......... .......... 91% 354M 0s 2019-12-09T11:38:37.198 INFO:teuthology.orchestra.run.smithi174.stderr: 67450K .......... .......... .......... .......... .......... 91% 335M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67500K .......... .......... .......... .......... .......... 91% 356M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67550K .......... .......... .......... .......... .......... 91% 205M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67600K .......... .......... .......... .......... .......... 91% 325M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67650K .......... .......... .......... .......... .......... 92% 401M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67700K .......... .......... .......... .......... .......... 92% 404M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67750K .......... .......... .......... .......... .......... 92% 221M 0s 2019-12-09T11:38:37.199 INFO:teuthology.orchestra.run.smithi174.stderr: 67800K .......... .......... .......... .......... .......... 92% 107M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 67850K .......... .......... .......... .......... .......... 92% 248M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 67900K .......... .......... .......... .......... .......... 92% 83.8M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 67950K .......... .......... .......... .......... .......... 92% 258M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 68000K .......... .......... .......... .......... .......... 92% 281M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 68050K .......... .......... .......... .......... .......... 92% 287M 0s 2019-12-09T11:38:37.200 INFO:teuthology.orchestra.run.smithi174.stderr: 68100K .......... .......... .......... .......... .......... 92% 89.4M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68150K .......... .......... .......... .......... .......... 92% 256M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68200K .......... .......... .......... .......... .......... 92% 83.4M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68250K .......... .......... .......... .......... .......... 92% 285M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68300K .......... .......... .......... .......... .......... 92% 298M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68350K .......... .......... .......... .......... .......... 92% 133M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68400K .......... .......... .......... .......... .......... 93% 337M 0s 2019-12-09T11:38:37.201 INFO:teuthology.orchestra.run.smithi174.stderr: 68450K .......... .......... .......... .......... .......... 93% 92.5M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68500K .......... .......... .......... .......... .......... 93% 349M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68550K .......... .......... .......... .......... .......... 93% 230M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68600K .......... .......... .......... .......... .......... 93% 84.5M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68650K .......... .......... .......... .......... .......... 93% 196M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68700K .......... .......... .......... .......... .......... 93% 287M 0s 2019-12-09T11:38:37.202 INFO:teuthology.orchestra.run.smithi174.stderr: 68750K .......... .......... .......... .......... .......... 93% 200M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 68800K .......... .......... .......... .......... .......... 93% 337M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 68850K .......... .......... .......... .......... .......... 93% 83.2M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 68900K .......... .......... .......... .......... .......... 93% 311M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 68950K .......... .......... .......... .......... .......... 93% 117M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 69000K .......... .......... .......... .......... .......... 93% 337M 0s 2019-12-09T11:38:37.203 INFO:teuthology.orchestra.run.smithi174.stderr: 69050K .......... .......... .......... .......... .......... 93% 336M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69100K .......... .......... .......... .......... .......... 94% 138M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69150K .......... .......... .......... .......... .......... 94% 273M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69200K .......... .......... .......... .......... .......... 94% 109M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69250K .......... .......... .......... .......... .......... 94% 388M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69300K .......... .......... .......... .......... .......... 94% 412M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69350K .......... .......... .......... .......... .......... 94% 180M 0s 2019-12-09T11:38:37.204 INFO:teuthology.orchestra.run.smithi174.stderr: 69400K .......... .......... .......... .......... .......... 94% 269M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69450K .......... .......... .......... .......... .......... 94% 120M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69500K .......... .......... .......... .......... .......... 94% 241M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69550K .......... .......... .......... .......... .......... 94% 240M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69600K .......... .......... .......... .......... .......... 94% 37.2M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69650K .......... .......... .......... .......... .......... 94% 293M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69700K .......... .......... .......... .......... .......... 94% 298M 0s 2019-12-09T11:38:37.205 INFO:teuthology.orchestra.run.smithi174.stderr: 69750K .......... .......... .......... .......... .......... 94% 253M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 69800K .......... .......... .......... .......... .......... 94% 261M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 69850K .......... .......... .......... .......... .......... 95% 293M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 69900K .......... .......... .......... .......... .......... 95% 294M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 69950K .......... .......... .......... .......... .......... 95% 238M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 70000K .......... .......... .......... .......... .......... 95% 294M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 70050K .......... .......... .......... .......... .......... 95% 289M 0s 2019-12-09T11:38:37.206 INFO:teuthology.orchestra.run.smithi174.stderr: 70100K .......... .......... .......... .......... .......... 95% 289M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70150K .......... .......... .......... .......... .......... 95% 247M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70200K .......... .......... .......... .......... .......... 95% 294M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70250K .......... .......... .......... .......... .......... 95% 295M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70300K .......... .......... .......... .......... .......... 95% 301M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70350K .......... .......... .......... .......... .......... 95% 228M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70400K .......... .......... .......... .......... .......... 95% 291M 0s 2019-12-09T11:38:37.207 INFO:teuthology.orchestra.run.smithi174.stderr: 70450K .......... .......... .......... .......... .......... 95% 295M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70500K .......... .......... .......... .......... .......... 95% 300M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70550K .......... .......... .......... .......... .......... 95% 256M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70600K .......... .......... .......... .......... .......... 96% 300M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70650K .......... .......... .......... .......... .......... 96% 81.5M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70700K .......... .......... .......... .......... .......... 96% 297M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70750K .......... .......... .......... .......... .......... 96% 239M 0s 2019-12-09T11:38:37.208 INFO:teuthology.orchestra.run.smithi174.stderr: 70800K .......... .......... .......... .......... .......... 96% 285M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 70850K .......... .......... .......... .......... .......... 96% 280M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 70900K .......... .......... .......... .......... .......... 96% 296M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 70950K .......... .......... .......... .......... .......... 96% 241M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 71000K .......... .......... .......... .......... .......... 96% 292M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 71050K .......... .......... .......... .......... .......... 96% 287M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 71100K .......... .......... .......... .......... .......... 96% 287M 0s 2019-12-09T11:38:37.209 INFO:teuthology.orchestra.run.smithi174.stderr: 71150K .......... .......... .......... .......... .......... 96% 11.8M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71200K .......... .......... .......... .......... .......... 96% 201M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71250K .......... .......... .......... .......... .......... 96% 228M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71300K .......... .......... .......... .......... .......... 96% 275M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71350K .......... .......... .......... .......... .......... 97% 223M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71400K .......... .......... .......... .......... .......... 97% 257M 0s 2019-12-09T11:38:37.210 INFO:teuthology.orchestra.run.smithi174.stderr: 71450K .......... .......... .......... .......... .......... 97% 269M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71500K .......... .......... .......... .......... .......... 97% 281M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71550K .......... .......... .......... .......... .......... 97% 245M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71600K .......... .......... .......... .......... .......... 97% 274M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71650K .......... .......... .......... .......... .......... 97% 276M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71700K .......... .......... .......... .......... .......... 97% 280M 0s 2019-12-09T11:38:37.211 INFO:teuthology.orchestra.run.smithi174.stderr: 71750K .......... .......... .......... .......... .......... 97% 246M 0s 2019-12-09T11:38:37.212 INFO:teuthology.orchestra.run.smithi174.stderr: 71800K .......... .......... .......... .......... .......... 97% 263M 0s 2019-12-09T11:38:37.212 INFO:teuthology.orchestra.run.smithi174.stderr: 71850K .......... .......... .......... .......... .......... 97% 289M 0s 2019-12-09T11:38:37.212 INFO:teuthology.orchestra.run.smithi174.stderr: 71900K .......... .......... .......... .......... .......... 97% 290M 0s 2019-12-09T11:38:37.212 INFO:teuthology.orchestra.run.smithi174.stderr: 71950K .......... .......... .......... .......... .......... 97% 87.3M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72000K .......... .......... .......... .......... .......... 97% 355M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72050K .......... .......... .......... .......... .......... 98% 116M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72100K .......... .......... .......... .......... .......... 98% 258M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72150K .......... .......... .......... .......... .......... 98% 34.3M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72200K .......... .......... .......... .......... .......... 98% 222M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72250K .......... .......... .......... .......... .......... 98% 290M 0s 2019-12-09T11:38:37.213 INFO:teuthology.orchestra.run.smithi174.stderr: 72300K .......... .......... .......... .......... .......... 98% 294M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72350K .......... .......... .......... .......... .......... 98% 249M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72400K .......... .......... .......... .......... .......... 98% 142M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72450K .......... .......... .......... .......... .......... 98% 208M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72500K .......... .......... .......... .......... .......... 98% 291M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72550K .......... .......... .......... .......... .......... 98% 17.1M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72600K .......... .......... .......... .......... .......... 98% 282M 0s 2019-12-09T11:38:37.214 INFO:teuthology.orchestra.run.smithi174.stderr: 72650K .......... .......... .......... .......... .......... 98% 293M 0s 2019-12-09T11:38:37.215 INFO:teuthology.orchestra.run.smithi174.stderr: 72700K .......... .......... .......... .......... .......... 98% 288M 0s 2019-12-09T11:38:37.215 INFO:teuthology.orchestra.run.smithi174.stderr: 72750K .......... .......... .......... .......... .......... 98% 233M 0s 2019-12-09T11:38:37.215 INFO:teuthology.orchestra.run.smithi174.stderr: 72800K .......... .......... .......... .......... .......... 99% 284M 0s 2019-12-09T11:38:37.215 INFO:teuthology.orchestra.run.smithi174.stderr: 72850K .......... .......... .......... .......... .......... 99% 291M 0s 2019-12-09T11:38:37.216 INFO:teuthology.orchestra.run.smithi174.stderr: 72900K .......... .......... .......... .......... .......... 99% 293M 0s 2019-12-09T11:38:37.216 INFO:teuthology.orchestra.run.smithi174.stderr: 72950K .......... .......... .......... .......... .......... 99% 244M 0s 2019-12-09T11:38:37.216 INFO:teuthology.orchestra.run.smithi174.stderr: 73000K .......... .......... .......... .......... .......... 99% 291M 0s 2019-12-09T11:38:37.216 INFO:teuthology.orchestra.run.smithi174.stderr: 73050K .......... .......... .......... .......... .......... 99% 276M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73100K .......... .......... .......... .......... .......... 99% 295M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73150K .......... .......... .......... .......... .......... 99% 268M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73200K .......... .......... .......... .......... .......... 99% 95.6M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73250K .......... .......... .......... .......... .......... 99% 284M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73300K .......... .......... .......... .......... .......... 99% 228M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73350K .......... .......... .......... .......... .......... 99% 233M 0s 2019-12-09T11:38:37.217 INFO:teuthology.orchestra.run.smithi174.stderr: 73400K .......... .......... .......... .......... .......... 99% 291M 0s 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr: 73450K .......... .......... .......... .......... .......... 99% 132M 0s 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr: 73500K .......... .......... .......... .......... .......... 99% 279M 0s 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr: 73550K ........ 100% 176M=0.4s 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09 11:38:35 (170 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-12-09T11:38:37.218 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout:Archive: gradle-4.7-bin.zip 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/ 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-12-09T11:38:37.219 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-12-09T11:38:37.220 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-12-09T11:38:37.221 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-12-09T11:38:37.222 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-12-09T11:38:37.223 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-12-09T11:38:37.224 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-12-09T11:38:37.225 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-12-09T11:38:37.226 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-12-09T11:38:37.227 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-12-09T11:38:37.228 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-12-09T11:38:37.229 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-12-09T11:38:37.230 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-12-09T11:38:37.231 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-12-09T11:38:37.232 INFO:teuthology.orchestra.run.smithi174.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-12-09T11:38:37.233 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-12-09T11:38:37.234 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-12-09T11:38:37.235 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-12-09T11:38:37.236 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-12-09T11:38:37.237 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-12-09T11:38:37.238 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-12-09T11:38:37.239 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-12-09T11:38:37.240 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-12-09T11:38:37.241 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-12-09T11:38:37.241 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-12-09T11:38:37.241 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-12-09T11:38:37.241 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-12-09T11:38:37.242 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-12-09T11:38:37.242 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-12-09T11:38:37.242 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-12-09T11:38:37.242 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-12-09T11:38:37.243 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-12-09T11:38:37.244 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-12-09T11:38:37.245 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-12-09T11:38:37.246 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-12-09T11:38:37.247 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-12-09T11:38:37.248 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-12-09T11:38:37.249 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-12-09T11:38:37.250 INFO:teuthology.orchestra.run.smithi174.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-12-09T11:38:37.251 INFO:teuthology.orchestra.run.smithi174.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-12-09T11:38:37.780 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:37.780 INFO:teuthology.orchestra.run.smithi174.stdout:Welcome to Gradle 4.7! 2019-12-09T11:38:37.780 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:37.780 INFO:teuthology.orchestra.run.smithi174.stdout:Here are the highlights of this release: 2019-12-09T11:38:37.780 INFO:teuthology.orchestra.run.smithi174.stdout: - Incremental annotation processing 2019-12-09T11:38:37.781 INFO:teuthology.orchestra.run.smithi174.stdout: - JDK 10 support 2019-12-09T11:38:37.781 INFO:teuthology.orchestra.run.smithi174.stdout: - Grouped non-interactive console logs 2019-12-09T11:38:37.781 INFO:teuthology.orchestra.run.smithi174.stdout: - Failed tests are re-run first for quicker feedback 2019-12-09T11:38:37.781 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:37.781 INFO:teuthology.orchestra.run.smithi174.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-12-09T11:38:37.782 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout:------------------------------------------------------------ 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout:Gradle 4.7 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout:------------------------------------------------------------ 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:38.106 INFO:teuthology.orchestra.run.smithi174.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout:Groovy: 2.4.12 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-12-09T11:38:38.107 INFO:teuthology.orchestra.run.smithi174.stdout:OS: Linux 3.10.0-1062.9.1.el7.x86_64 amd64 2019-12-09T11:38:38.108 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:38.112 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-12-09T11:38:38.113 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-12-09T11:38:38.113 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi106.front.sepia.ceph.com 2019-12-09T11:38:38.113 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-12-09T11:38:38.113 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi174.front.sepia.ceph.com 2019-12-09T11:38:38.113 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-12-09T11:38:38.522 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-12-09T11:38:38.523 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-12-09T11:38:38.523 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'YGOHFCIYOUHOVOBYIJUW', 's3dir': 'folder', 'endpoint': 'smithi174.front.sepia.ceph.com:80', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 80, 'access_secret': 'RFt7rdz5SKKKcUEbLR/TmhPDAzvvB2kIjB8E/4P6PHoWvsovedq6xA==', 'host': 'smithi174.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': False, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-12-09T11:38:38.523 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-12-09T11:38:38.523 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'YGOHFCIYOUHOVOBYIJUW', '--secret', 'RFt7rdz5SKKKcUEbLR/TmhPDAzvvB2kIjB8E/4P6PHoWvsovedq6xA==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-12-09T11:38:38.524 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key YGOHFCIYOUHOVOBYIJUW --secret RFt7rdz5SKKKcUEbLR/TmhPDAzvvB2kIjB8E/4P6PHoWvsovedq6xA== --email tester.client.0_test@test.test --cluster ceph 2019-12-09T11:38:38.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.720+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.755+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:38.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.755+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.755 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.763+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:38.755 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.763+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.771+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:38.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.771+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.785+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:38.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.938+0000 7f7e06882ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:38:38.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.938+0000 7f7e06882ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:38:38.931 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.938+0000 7f7e06882ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:38:38.931 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.938+0000 7f7e06882ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:38:38.931 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.938+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.938 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.946+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:38.938 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.946+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.953 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.961+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:38.953 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.961+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.974+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:38.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.974+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.984+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:38.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.984+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.990+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:38.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.990+0000 7f7e06882ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:38:38.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.990+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:38.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.998+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:38.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:38.998+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.004 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.012+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:39.004 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.012+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.019+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:39.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.019+0000 7f7e06882ec0 20 zonegroup default 2019-12-09T11:38:39.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.019+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.017 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.025+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:39.018 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.025+0000 7f7e06882ec0 10 Cannot find current period zone using local zone 2019-12-09T11:38:39.018 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.025+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.034+0000 7f7e06882ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:39.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.034+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.033 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.041+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:39.033 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.041+0000 7f7e06882ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:39.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.053+0000 7f7e06882ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:39.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.053+0000 7f7e06882ec0 20 zone default found 2019-12-09T11:38:39.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.053+0000 7f7e06882ec0 20 started sync module instance, tier type = 2019-12-09T11:38:39.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.053+0000 7f7e06882ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:38:39.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=0 2019-12-09T11:38:39.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=1 2019-12-09T11:38:39.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=2 2019-12-09T11:38:39.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=3 2019-12-09T11:38:39.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=4 2019-12-09T11:38:39.451 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=5 2019-12-09T11:38:39.451 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.457+0000 7f7e06882ec0 20 add_watcher() i=6 2019-12-09T11:38:39.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.492+0000 7f7e06882ec0 20 add_watcher() i=7 2019-12-09T11:38:39.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.492+0000 7f7e06882ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:38:39.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.492+0000 7f7d47fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:38:39.486 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.494+0000 7f7debfff700 0 -- 172.21.15.174:0/2490995584 >> v2:172.21.15.174:3300/0 conn(0x5634bf5b8560 msgr2=0x5634bf5baa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T11:38:39.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:39.540+0000 7f7df3aac700 0 -- 172.21.15.174:0/3318909569 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f7d34022480 msgr2=0x7f7d34024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:38:40.977 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.984+0000 7f7d327fc700 20 reqs_thread_entry: start 2019-12-09T11:38:40.977 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.985+0000 7f7d2bfff700 20 reqs_thread_entry: start 2019-12-09T11:38:40.977 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.985+0000 7f7e06882ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:38:40.978 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.985+0000 7f7d2a7fc700 20 reqs_thread_entry: start 2019-12-09T11:38:40.978 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.985+0000 7f7e06882ec0 20 get_system_obj_state: rctx=0x7ffde0c87578 obj=default.rgw.meta:users.uid:tester.client.0 state=0x5634bf633070 s->prefetch_data=0 2019-12-09T11:38:40.978 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.985+0000 7f7e06882ec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-12-09T11:38:40.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.994+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-12-09T11:38:40.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.994+0000 7f7e06882ec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T11:38:40.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.994+0000 7f7e06882ec0 20 get_system_obj_state: rctx=0x7ffde0c87588 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x5634bf634e00 s->prefetch_data=0 2019-12-09T11:38:40.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.994+0000 7f7e06882ec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-12-09T11:38:40.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.999+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-12-09T11:38:40.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.999+0000 7f7e06882ec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T11:38:40.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.999+0000 7f7e06882ec0 20 get_system_obj_state: rctx=0x7ffde0c87588 obj=default.rgw.meta:users.keys:YGOHFCIYOUHOVOBYIJUW state=0x5634bf634e00 s->prefetch_data=0 2019-12-09T11:38:40.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:40.999+0000 7f7e06882ec0 10 cache get: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW : miss 2019-12-09T11:38:40.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.005+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW info.flags=0x0 2019-12-09T11:38:40.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.005+0000 7f7e06882ec0 10 adding default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW to cache LRU end 2019-12-09T11:38:40.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.005+0000 7f7e06882ec0 20 get_system_obj_state: rctx=0x7ffde0c87198 obj=default.rgw.meta:users.keys:YGOHFCIYOUHOVOBYIJUW state=0x5634bf634e00 s->prefetch_data=0 2019-12-09T11:38:40.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.005+0000 7f7e06882ec0 10 cache get: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW : type miss (requested=0x6, cached=0x0) 2019-12-09T11:38:41.004 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.012+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW info.flags=0x0 2019-12-09T11:38:41.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.012+0000 7f7e06882ec0 10 moving default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW to cache LRU end 2019-12-09T11:38:41.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.012+0000 7f7e06882ec0 20 get_system_obj_state: rctx=0x7ffde0c87318 obj=default.rgw.meta:users.keys:YGOHFCIYOUHOVOBYIJUW state=0x5634bf634e00 s->prefetch_data=0 2019-12-09T11:38:41.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.012+0000 7f7e06882ec0 10 cache get: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW : type miss (requested=0x6, cached=0x0) 2019-12-09T11:38:41.010 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.018+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW info.flags=0x0 2019-12-09T11:38:41.010 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.018+0000 7f7e06882ec0 10 moving default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW to cache LRU end 2019-12-09T11:38:41.068 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.076+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T11:38:41.069 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.076+0000 7f7e06882ec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T11:38:41.069 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.076+0000 7f7e06882ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-12-09T11:38:41.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.092+0000 7f7de97fa700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94784548644192 notifier 4523 bl.length()=632 2019-12-09T11:38:41.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.092+0000 7f7de97fa700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-12-09T11:38:41.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.092+0000 7f7de97fa700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-12-09T11:38:41.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.608+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T11:38:41.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.608+0000 7f7e06882ec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T11:38:41.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.608+0000 7f7e06882ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-12-09T11:38:41.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.615+0000 7f7de97fa700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94784548637936 notifier 4523 bl.length()=200 2019-12-09T11:38:41.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.615+0000 7f7de97fa700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-12-09T11:38:41.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.615+0000 7f7de97fa700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-12-09T11:38:41.709 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.716+0000 7f7e06882ec0 10 cache put: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW info.flags=0x7 2019-12-09T11:38:41.709 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.716+0000 7f7e06882ec0 10 moving default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW to cache LRU end 2019-12-09T11:38:41.709 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.716+0000 7f7e06882ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=189 2019-12-09T11:38:41.723 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.731+0000 7f7de97fa700 10 RGWWatcher::handle_notify() notify_id 579820584974 cookie 94784548639488 notifier 4523 bl.length()=189 2019-12-09T11:38:41.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.731+0000 7f7de97fa700 10 cache put: name=default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW info.flags=0x7 2019-12-09T11:38:41.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:41.731+0000 7f7de97fa700 10 moving default.rgw.meta+users.keys+YGOHFCIYOUHOVOBYIJUW to cache LRU end 2019-12-09T11:38:42.036 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T11:38:42.036 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "tester.client.0", 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Ms. tester.client.0", 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "tester.client.0_test@test.test", 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T11:38:42.037 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "tester.client.0", 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "YGOHFCIYOUHOVOBYIJUW", 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "RFt7rdz5SKKKcUEbLR/TmhPDAzvvB2kIjB8E/4P6PHoWvsovedq6xA==" 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T11:38:42.038 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:38:42.039 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T11:38:42.040 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T11:38:42.041 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T11:38:42.042 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T11:38:42.042 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:38:42.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.108+0000 7f7e06882ec0 20 remove_watcher() i=0 2019-12-09T11:38:42.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.108+0000 7f7e06882ec0 2 removed watcher, disabling cache 2019-12-09T11:38:42.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.172+0000 7f7e06882ec0 20 remove_watcher() i=1 2019-12-09T11:38:42.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.244+0000 7f7e06882ec0 20 remove_watcher() i=2 2019-12-09T11:38:42.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.311+0000 7f7e06882ec0 20 remove_watcher() i=3 2019-12-09T11:38:42.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.364+0000 7f7e06882ec0 20 remove_watcher() i=4 2019-12-09T11:38:42.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.411+0000 7f7e06882ec0 20 remove_watcher() i=5 2019-12-09T11:38:42.446 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.454+0000 7f7e06882ec0 20 remove_watcher() i=6 2019-12-09T11:38:42.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.487+0000 7f7e06882ec0 20 remove_watcher() i=7 2019-12-09T11:38:42.488 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-12-09T11:38:42.544 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-12-09T11:38:42.545 INFO:teuthology.orchestra.run.smithi174:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-12-09T11:38:42.639 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T11:38:42.759 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.767+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:42.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.794+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:42.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.795+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:42.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.804+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:42.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.804+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:42.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.813+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:42.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.813+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:42.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:42.834+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:43.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.026+0000 7f2e813efec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:38:43.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.026+0000 7f2e813efec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:38:43.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.026+0000 7f2e813efec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:38:43.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.026+0000 7f2e813efec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:38:43.020 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.026+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.031 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.039+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:43.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.039+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.043 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.050+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:43.043 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.050+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.067+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:43.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.067+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.079+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:43.071 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.079+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.082 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.090+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:43.082 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.090+0000 7f2e813efec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:38:43.083 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.090+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.093 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.101+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:43.093 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.101+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.111 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.119+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:43.111 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.119+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.126+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:43.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.126+0000 7f2e813efec0 20 zonegroup default 2019-12-09T11:38:43.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.126+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.135+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:43.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.135+0000 7f2e813efec0 10 Cannot find current period zone using local zone 2019-12-09T11:38:43.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.135+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.136 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.143+0000 7f2e813efec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:43.136 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.143+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.159+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.159+0000 7f2e813efec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:43.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.171+0000 7f2e813efec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:43.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.171+0000 7f2e813efec0 20 zone default found 2019-12-09T11:38:43.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.171+0000 7f2e813efec0 20 started sync module instance, tier type = 2019-12-09T11:38:43.165 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.171+0000 7f2e813efec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:38:43.500 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=0 2019-12-09T11:38:43.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=1 2019-12-09T11:38:43.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=2 2019-12-09T11:38:43.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=3 2019-12-09T11:38:43.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=4 2019-12-09T11:38:43.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.508+0000 7f2e813efec0 20 add_watcher() i=5 2019-12-09T11:38:43.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.509+0000 7f2e813efec0 20 add_watcher() i=6 2019-12-09T11:38:43.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.539+0000 7f2e813efec0 20 add_watcher() i=7 2019-12-09T11:38:43.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.539+0000 7f2e813efec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:38:43.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.539+0000 7f2e3efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:38:43.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.551+0000 7f2e6e619700 0 -- 172.21.15.174:0/1418157208 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x5577b1d46d20 msgr2=0x5577b1d47130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:38:43.594 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:43.601+0000 7f2e6de18700 0 -- 172.21.15.174:0/3736554805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5577b1d1fe60 msgr2=0x5577b1d586e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T11:38:44.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:44.975+0000 7f2dad7fa700 20 reqs_thread_entry: start 2019-12-09T11:38:44.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:44.976+0000 7f2da6ffd700 20 reqs_thread_entry: start 2019-12-09T11:38:44.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:44.976+0000 7f2e813efec0 20 init_complete bucket index max shards: 0 2019-12-09T11:38:44.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:44.976+0000 7f2da57fa700 20 reqs_thread_entry: start 2019-12-09T11:38:44.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:44.976+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T11:38:45.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.048+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:45.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.071+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:45.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:45.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.55', time=2019-12-09T13:38:07.039612+0000, chain.objs.size()=4 2019-12-09T11:38:45.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4__multipart_foo.2~PubrTtFefNBxFCnZ9tAj6Gg-MT9JAt3.1 2019-12-09T11:38:45.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4__multipart_foo.2~PubrTtFefNBxFCnZ9tAj6Gg-MT9JAt3.2 2019-12-09T11:38:45.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4__multipart_foo.2~PubrTtFefNBxFCnZ9tAj6Gg-MT9JAt3.3 2019-12-09T11:38:45.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.107+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.4__multipart_foo.2~PubrTtFefNBxFCnZ9tAj6Gg-MT9JAt3.4 2019-12-09T11:38:45.227 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.235+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:45.362 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.370+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T11:38:45.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.418+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:45.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.434+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:45.438 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.203', time=2019-12-09T13:38:18.149304+0000, chain.objs.size()=5 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-LUKEWARM.2~9ZZ9Qb6-Eqwcq6odyC3pMxdGFzTLw0n.1 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__shadow_foo-LUKEWARM.2~9ZZ9Qb6-Eqwcq6odyC3pMxdGFzTLw0n.1_1 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-LUKEWARM.2~9ZZ9Qb6-Eqwcq6odyC3pMxdGFzTLw0n.2 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__shadow_foo-LUKEWARM.2~9ZZ9Qb6-Eqwcq6odyC3pMxdGFzTLw0n.2_1 2019-12-09T11:38:45.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.446+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-LUKEWARM.2~9ZZ9Qb6-Eqwcq6odyC3pMxdGFzTLw0n.3 2019-12-09T11:38:45.618 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.626+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:45.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.743+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T11:38:45.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.792+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:45.799 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.807+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:45.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.836+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:45.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.836+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.109', time=2019-12-09T13:38:10.504424+0000, chain.objs.size()=2 2019-12-09T11:38:45.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.836+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.FIXJegHuvoQELbxVOoZy5fw4TLUZb74_0 2019-12-09T11:38:45.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.836+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.FIXJegHuvoQELbxVOoZy5fw4TLUZb74_1 2019-12-09T11:38:45.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:45.888+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:46.015 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.023+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T11:38:46.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.081+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:46.089 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.097+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:46.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:46.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_j1yPZ8nkWMNyN053bUq4J_TNQz_Mfj0', time=2019-12-09T13:38:11.265547+0000, chain.objs.size()=2 2019-12-09T11:38:46.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.C3Xn52X9FxZSxgdM-l0h3DdS86YSdq9_0 2019-12-09T11:38:46.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.C3Xn52X9FxZSxgdM-l0h3DdS86YSdq9_1 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.225', time=2019-12-09T13:38:13.929490+0000, chain.objs.size()=5 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.1 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.1_1 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.2 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.2_1 2019-12-09T11:38:46.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.136+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.3 2019-12-09T11:38:46.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.334+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:46.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.491+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T11:38:46.574 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.582+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:46.596 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.605+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:46.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.617+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:46.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.665+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:38:46.753 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.762+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:46.774 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.782+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:46.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.811+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:46.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.811+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_aDfXTllXMftoD6j3Qq9ZQUF1RP9fZQL', time=2019-12-09T13:38:12.184917+0000, chain.objs.size()=1 2019-12-09T11:38:46.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.811+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.E5aUAcDF6YdyIBRlZ_KOt9aqg6dwCgh_0 2019-12-09T11:38:46.876 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:46.884+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:47.049 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.058+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:38:47.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.138+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:47.167 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.175+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.198+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.198+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_DUai0F0zcEH7aFWd4xPvrxDw6bgBKiF', time=2019-12-09T13:38:15.714675+0000, chain.objs.size()=1 2019-12-09T11:38:47.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.198+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.k5OF3JuXc5_V6q9iN2jPSP6nYC5WulW_0 2019-12-09T11:38:47.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.269+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:47.342 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.350+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:38:47.441 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.449+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:47.464 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.473+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:47.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.488+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:47.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.488+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_CwCU1s1pGcbJMWNZG7VZmw8JSB5fR3P', time=2019-12-09T13:38:15.316102+0000, chain.objs.size()=2 2019-12-09T11:38:47.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.488+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.HZOUTVBA366V1Ozjh5qtRby6mI8Djpl_1 2019-12-09T11:38:47.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.488+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.HZOUTVBA366V1Ozjh5qtRby6mI8Djpl_2 2019-12-09T11:38:47.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.577+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:47.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.692+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:38:47.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.795+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:47.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.829+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:47.867 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_ogy-Je_f93SfK-4gI6H1CcBVnWZdJoH', time=2019-12-09T13:38:14.442328+0000, chain.objs.size()=2 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.EzAjU1xzwhSPH23XkRmMcu8Zd3-5OEC_1 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.EzAjU1xzwhSPH23XkRmMcu8Zd3-5OEC_2 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.85', time=2019-12-09T13:38:19.578852+0000, chain.objs.size()=3 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.Bmn6X3zxZ0GnDwRIscC__9-BaZFil0d_0 2019-12-09T11:38:47.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.Bmn6X3zxZ0GnDwRIscC__9-BaZFil0d_1 2019-12-09T11:38:47.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:47.875+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.Bmn6X3zxZ0GnDwRIscC__9-BaZFil0d_2 2019-12-09T11:38:48.061 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.070+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:48.179 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.187+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:38:48.225 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.233+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:48.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.248+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.261+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.261+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.233', time=2019-12-09T13:38:14.634038+0000, chain.objs.size()=5 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.261+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.1 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.261+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.1_1 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.261+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.2 2019-12-09T11:38:48.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.2_1 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.3 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.235', time=2019-12-09T13:38:15.086177+0000, chain.objs.size()=5 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.1 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.1_1 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.2 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.2_1 2019-12-09T11:38:48.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.262+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-LUKEWARM.2~SvaahVdDTvkmBPnUhcu8YY7uqHvY49x.3 2019-12-09T11:38:48.551 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.559+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:48.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.642+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T11:38:48.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.692+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:48.699 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.708+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:48.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:48.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.245', time=2019-12-09T13:38:16.176234+0000, chain.objs.size()=3 2019-12-09T11:38:48.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.1 2019-12-09T11:38:48.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.2 2019-12-09T11:38:48.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.3 2019-12-09T11:38:48.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.196', time=2019-12-09T13:38:17.928828+0000, chain.objs.size()=5 2019-12-09T11:38:48.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-FROZEN.2~ycFHPHTsEshR2mZl1zFyT2UlOWe1xbd.1 2019-12-09T11:38:48.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__shadow_foo-FROZEN.2~ycFHPHTsEshR2mZl1zFyT2UlOWe1xbd.1_1 2019-12-09T11:38:48.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-FROZEN.2~ycFHPHTsEshR2mZl1zFyT2UlOWe1xbd.2 2019-12-09T11:38:48.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__shadow_foo-FROZEN.2~ycFHPHTsEshR2mZl1zFyT2UlOWe1xbd.2_1 2019-12-09T11:38:48.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.742+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-FROZEN.2~ycFHPHTsEshR2mZl1zFyT2UlOWe1xbd.3 2019-12-09T11:38:48.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:48.907+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:49.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.048+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T11:38:49.089 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.098+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:49.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.122+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:49.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.138+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:49.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.138+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='__aYit9ZDRdeeE85eAjbezHXKSCnz8Ki', time=2019-12-09T13:38:14.107987+0000, chain.objs.size()=3 2019-12-09T11:38:49.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.138+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.XqCfTZEcrLCcj2dRP4zXhO34QAe2bzU_0 2019-12-09T11:38:49.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.138+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.XqCfTZEcrLCcj2dRP4zXhO34QAe2bzU_1 2019-12-09T11:38:49.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.138+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.XqCfTZEcrLCcj2dRP4zXhO34QAe2bzU_2 2019-12-09T11:38:49.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.387+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:49.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.542+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T11:38:49.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.599+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:49.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.623+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:49.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:49.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.71', time=2019-12-09T13:38:08.822278+0000, chain.objs.size()=4 2019-12-09T11:38:49.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5__multipart_foo.2~bV1WEXPK5Z1lgm0ScjchAtnKhYuoN_8.1 2019-12-09T11:38:49.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5__multipart_foo.2~bV1WEXPK5Z1lgm0ScjchAtnKhYuoN_8.2 2019-12-09T11:38:49.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5__multipart_foo.2~bV1WEXPK5Z1lgm0ScjchAtnKhYuoN_8.3 2019-12-09T11:38:49.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.637+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.5__multipart_foo.2~bV1WEXPK5Z1lgm0ScjchAtnKhYuoN_8.4 2019-12-09T11:38:49.771 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.779+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:49.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.881+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T11:38:49.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.947+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:49.960 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.968+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:49.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:49.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.20', time=2019-12-09T13:38:04.161173+0000, chain.objs.size()=4 2019-12-09T11:38:49.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2__multipart_foo.2~vPPAZ8CFdyUrmam1VJRFi4n9xa4r9Tc.1 2019-12-09T11:38:49.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2__multipart_foo.2~vPPAZ8CFdyUrmam1VJRFi4n9xa4r9Tc.2 2019-12-09T11:38:49.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2__multipart_foo.2~vPPAZ8CFdyUrmam1VJRFi4n9xa4r9Tc.3 2019-12-09T11:38:49.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.2__multipart_foo.2~vPPAZ8CFdyUrmam1VJRFi4n9xa4r9Tc.4 2019-12-09T11:38:49.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_-gTh5aNFxZnRNx1cE_abbxQz7vSxeFm', time=2019-12-09T13:38:12.027768+0000, chain.objs.size()=1 2019-12-09T11:38:49.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:49.981+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.KKPi1FU-2-e19oMPCmuZIcl5xaI94W6_0 2019-12-09T11:38:50.126 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.134+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:50.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.225+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T11:38:50.263 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.271+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:50.285 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.293+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:50.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.305+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:50.349 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.357+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T11:38:50.438 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.446+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:50.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.477+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:50.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.495+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:50.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.495+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_1Q4UA6FhCSVLNqMp4rXpKtLhMYFHPby', time=2019-12-09T13:38:14.890612+0000, chain.objs.size()=3 2019-12-09T11:38:50.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.495+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.ZZ9dnyCGAm7dbVsQ4ne5W8ar7LGauFg_0 2019-12-09T11:38:50.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.495+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.ZZ9dnyCGAm7dbVsQ4ne5W8ar7LGauFg_1 2019-12-09T11:38:50.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.495+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.ZZ9dnyCGAm7dbVsQ4ne5W8ar7LGauFg_2 2019-12-09T11:38:50.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.726+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:50.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.834+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T11:38:50.867 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.875+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:50.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.890+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:50.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.899+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:50.932 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:50.940+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T11:38:51.004 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.012+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.044+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:51.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.059+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.095 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.103+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T11:38:51.143 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.151+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.160 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.168+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:51.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.180+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:51.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.180+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.108', time=2019-12-09T13:38:10.313493+0000, chain.objs.size()=2 2019-12-09T11:38:51.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.180+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.FIXJegHuvoQELbxVOoZy5fw4TLUZb74_0 2019-12-09T11:38:51.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.180+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.FIXJegHuvoQELbxVOoZy5fw4TLUZb74_1 2019-12-09T11:38:51.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.354+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:51.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.431+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T11:38:51.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.553+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.590+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:51.597 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.605+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.650+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T11:38:51.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.721+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:51.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.745+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:51.777 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:51.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.40', time=2019-12-09T13:38:05.424504+0000, chain.objs.size()=4 2019-12-09T11:38:51.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3__multipart_foo.2~6l9F2dDAsRsi0OwmrsqHUeq6PKSt-bi.1 2019-12-09T11:38:51.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3__multipart_foo.2~6l9F2dDAsRsi0OwmrsqHUeq6PKSt-bi.2 2019-12-09T11:38:51.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3__multipart_foo.2~6l9F2dDAsRsi0OwmrsqHUeq6PKSt-bi.3 2019-12-09T11:38:51.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.3__multipart_foo.2~6l9F2dDAsRsi0OwmrsqHUeq6PKSt-bi.4 2019-12-09T11:38:51.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_U5pfPmLToH1H9IhI9XLLgor8Yr8Hu8M', time=2019-12-09T13:38:10.726326+0000, chain.objs.size()=2 2019-12-09T11:38:51.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.RMtvD7Bmni2cLewSWZnL68m1qp1t4sM_0 2019-12-09T11:38:51.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.785+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.RMtvD7Bmni2cLewSWZnL68m1qp1t4sM_1 2019-12-09T11:38:51.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:51.963+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:52.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.225+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T11:38:52.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.269+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:52.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.286+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:52.287 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.296+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:52.330 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.339+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T11:38:52.382 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.390+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:52.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.407+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:52.411 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.419+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:52.411 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.419+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_0b2r_baSRIzmLiIQiIeJceAAumH-pur', time=2019-12-09T13:38:10.921035+0000, chain.objs.size()=1 2019-12-09T11:38:52.411 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.419+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.mCfIJz6sE4x4vPzk7n3FTBCAA6n_CTE_1 2019-12-09T11:38:52.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.512+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:52.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.646+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T11:38:52.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.706+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:52.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.729+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:52.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.744+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:52.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.744+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.210', time=2019-12-09T13:38:18.376124+0000, chain.objs.size()=3 2019-12-09T11:38:52.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.744+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-STANDARD.2~1yqEhHdtVsX5m6I-HPzK5n1iDl4-JhA.1 2019-12-09T11:38:52.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.744+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-STANDARD.2~1yqEhHdtVsX5m6I-HPzK5n1iDl4-JhA.2 2019-12-09T11:38:52.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.744+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.8__multipart_foo-STANDARD.2~1yqEhHdtVsX5m6I-HPzK5n1iDl4-JhA.3 2019-12-09T11:38:52.872 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.879+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:52.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:52.959+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T11:38:52.996 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.004+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.027+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2019-12-09T11:38:53.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.114', time=2019-12-09T13:38:11.447745+0000, chain.objs.size()=2 2019-12-09T11:38:53.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.ao6APf9NEPAB1dxeWGk21cT4wsleh3i_0 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.ao6APf9NEPAB1dxeWGk21cT4wsleh3i_1 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.224', time=2019-12-09T13:38:13.762623+0000, chain.objs.size()=5 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.1 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.1_1 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.2 2019-12-09T11:38:53.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.2_1 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-FROZEN.2~WxRjYGgN93WrvlpXC8Hek9y31o6T0Y4.3 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_loTPLdPFdXl7iD3eWBCQTbyl_SNfNNp', time=2019-12-09T13:38:15.899020+0000, chain.objs.size()=1 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__shadow_.p8GyzkBvkOJRzAarxj3z3uf9kvLtQWP_0 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.84', time=2019-12-09T13:38:19.385402+0000, chain.objs.size()=3 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.LvMYEM6SDSJ8YfLl4vcvEIeim0fx4KD_0 2019-12-09T11:38:53.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.LvMYEM6SDSJ8YfLl4vcvEIeim0fx4KD_1 2019-12-09T11:38:53.039 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.043+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.6__shadow_.LvMYEM6SDSJ8YfLl4vcvEIeim0fx4KD_2 2019-12-09T11:38:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.305+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:53.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.388+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T11:38:53.420 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.428+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.442 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.451+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.451 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.459+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.493+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T11:38:53.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.537+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.549 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.558+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.570+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.597 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.605+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T11:38:53.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.662+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.670 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.679+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.679 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.688+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.710 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.718+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T11:38:53.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.760+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.776+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.777 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.785+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.822+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T11:38:53.889 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.897+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:53.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.917+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.112', time=2019-12-09T13:38:11.113709+0000, chain.objs.size()=2 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.ao6APf9NEPAB1dxeWGk21cT4wsleh3i_0 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.ao6APf9NEPAB1dxeWGk21cT4wsleh3i_1 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='_KIvkhsqD44Xg278QRk3YBykaJduTt7a', time=2019-12-09T13:38:11.691913+0000, chain.objs.size()=1 2019-12-09T11:38:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:53.931+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.7__shadow_.rAP3ZLK-7m7ELDobCw1E0DYaoVKhHjG_1 2019-12-09T11:38:54.106 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.114+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:54.192 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.200+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T11:38:54.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.248+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:54.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.264+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:54.264 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.272+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:54.307 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.315+0000 7f2e813efec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T11:38:54.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.365+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:54.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.387+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:54.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.406+0000 7f2e813efec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T11:38:54.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.406+0000 7f2e813efec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.242', time=2019-12-09T13:38:15.551333+0000, chain.objs.size()=3 2019-12-09T11:38:54.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.406+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.1 2019-12-09T11:38:54.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.406+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.2 2019-12-09T11:38:54.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.406+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.9__multipart_foo-STANDARD.2~zgbKKLWw0tBgSlNyjGrOliOqk_Possq.3 2019-12-09T11:38:54.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.488+0000 7f2e813efec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T11:38:54.594 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.602+0000 7f2e813efec0 20 remove_watcher() i=0 2019-12-09T11:38:54.594 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.602+0000 7f2e813efec0 2 removed watcher, disabling cache 2019-12-09T11:38:54.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.663+0000 7f2e813efec0 20 remove_watcher() i=1 2019-12-09T11:38:54.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.702+0000 7f2e813efec0 20 remove_watcher() i=2 2019-12-09T11:38:54.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.760+0000 7f2e813efec0 20 remove_watcher() i=3 2019-12-09T11:38:54.795 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.803+0000 7f2e813efec0 20 remove_watcher() i=4 2019-12-09T11:38:54.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.848+0000 7f2e813efec0 20 remove_watcher() i=5 2019-12-09T11:38:54.875 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.883+0000 7f2e813efec0 20 remove_watcher() i=6 2019-12-09T11:38:54.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:54.920+0000 7f2e813efec0 20 remove_watcher() i=7 2019-12-09T11:38:54.919 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T11:38:55.026 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.034+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.053 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.061+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.053 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.061+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.070+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.070+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.080+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.080+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.093 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.101+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:55.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.297+0000 7ff56f1b2ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T11:38:55.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.297+0000 7ff56f1b2ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T11:38:55.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.297+0000 7ff56f1b2ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T11:38:55.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.297+0000 7ff56f1b2ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T11:38:55.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.297+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.301 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.310+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:55.302 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.310+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.319+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:55.312 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.320+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.334+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:55.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.334+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.346+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:55.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.346+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.355+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.355+0000 7ff56f1b2ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T11:38:55.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.355+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.364+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.364+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.368 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.376+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:55.369 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.376+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.396+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T11:38:55.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.396+0000 7ff56f1b2ec0 20 zonegroup default 2019-12-09T11:38:55.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.396+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.403+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.403+0000 7ff56f1b2ec0 10 Cannot find current period zone using local zone 2019-12-09T11:38:55.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.403+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.404 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.412+0000 7ff56f1b2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T11:38:55.404 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.412+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.424+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T11:38:55.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.424+0000 7ff56f1b2ec0 20 rados->read ofs=0 len=0 2019-12-09T11:38:55.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.439+0000 7ff56f1b2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T11:38:55.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.439+0000 7ff56f1b2ec0 20 zone default found 2019-12-09T11:38:55.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.439+0000 7ff56f1b2ec0 20 started sync module instance, tier type = 2019-12-09T11:38:55.432 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.439+0000 7ff56f1b2ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T11:38:55.739 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=0 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=1 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=2 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=3 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=4 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=5 2019-12-09T11:38:55.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.741+0000 7ff56f1b2ec0 20 add_watcher() i=6 2019-12-09T11:38:55.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.769+0000 7ff56f1b2ec0 20 add_watcher() i=7 2019-12-09T11:38:55.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.769+0000 7ff56f1b2ec0 2 all 8 watchers are set, enabling cache 2019-12-09T11:38:55.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.769+0000 7ff52cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T11:38:55.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.772+0000 7ff55c3dc700 0 -- >> v2:172.21.15.174:3300/0 conn(0x55a7ad4c04b0 msgr2=0x55a7ad4c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T11:38:55.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:55.805+0000 7ff55c3dc700 0 -- 172.21.15.174:0/663197779 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7ff4a0022470 msgr2=0x7ff4a0024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T11:38:57.330 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.335+0000 7ff49effd700 20 reqs_thread_entry: start 2019-12-09T11:38:57.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.336+0000 7ff49cff9700 20 reqs_thread_entry: start 2019-12-09T11:38:57.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.336+0000 7ff56f1b2ec0 20 init_complete bucket index max shards: 0 2019-12-09T11:38:57.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.337+0000 7ff492ffd700 20 reqs_thread_entry: start 2019-12-09T11:38:57.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.337+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T11:38:57.396 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.403+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.425+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:57.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.435+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.476+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T11:38:57.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.520+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.541+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:57.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.552+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.579+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T11:38:57.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.632+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.651+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:57.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.660+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.695 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.702+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T11:38:57.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.743+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.755 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.762+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:57.764 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.771+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.819+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T11:38:57.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.898+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.920+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:57.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.934+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:57.967 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:57.974+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T11:38:58.023 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.030+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.047 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.054+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.060 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.067+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.117+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T11:38:58.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.185+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.199 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.206+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.217+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.267+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T11:38:58.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.331+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.354+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.368+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.402+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T11:38:58.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.476+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.509+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.522+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.564+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T11:38:58.606 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.613+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.623 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.630+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.634 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.641+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.672 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.679+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T11:38:58.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.724+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.743+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.753+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.793+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T11:38:58.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.844+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.861+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.864 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.871+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.913+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T11:38:58.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.963+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:58.972 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.979+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:58.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:58.989+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.028+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T11:38:59.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.105+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.137+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.142 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.149+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.208+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T11:38:59.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.273+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.296+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.301 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.308+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.352 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.359+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T11:38:59.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.431+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.456+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.465 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.472+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.516 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.523+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T11:38:59.577 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.584+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.595 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.602+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.612+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.650+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T11:38:59.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.744+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.756 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.763+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.775+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.823+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T11:38:59.881 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.887+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:38:59.916 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.923+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:38:59.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:38:59.941+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.012+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T11:39:00.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.069+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.086+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.090 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.097+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.125 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.132+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T11:39:00.180 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.187+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.199 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.206+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.217+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.256+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T11:39:00.332 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.339+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.362+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.374+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.409 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.416+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T11:39:00.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.501+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.517 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.524+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.541+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.742+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T11:39:00.810 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.817+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.834 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.841+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.852+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.896 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.903+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T11:39:00.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.949+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:00.966 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.973+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:00.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:00.981+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.010 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.016+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T11:39:01.065 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.072+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.091+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.096 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.103+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.139 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.145+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T11:39:01.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.209+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.228+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.233 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.237+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.272+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T11:39:01.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.348+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.369 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.376+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.382 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.389+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.443+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T11:39:01.503 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.510+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.530+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.534 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.541+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.587+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T11:39:01.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.634+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.648 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.655+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.663+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.725+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T11:39:01.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.803+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.829+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.836 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.843+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.885 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.892+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T11:39:01.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.949+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:01.960 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.967+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T11:39:01.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:01.976+0000 7ff56f1b2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T11:39:02.057 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.064+0000 7ff56f1b2ec0 20 remove_watcher() i=0 2019-12-09T11:39:02.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.064+0000 7ff56f1b2ec0 2 removed watcher, disabling cache 2019-12-09T11:39:02.109 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.116+0000 7ff56f1b2ec0 20 remove_watcher() i=1 2019-12-09T11:39:02.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.174+0000 7ff56f1b2ec0 20 remove_watcher() i=2 2019-12-09T11:39:02.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.236+0000 7ff56f1b2ec0 20 remove_watcher() i=3 2019-12-09T11:39:02.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.304+0000 7ff56f1b2ec0 20 remove_watcher() i=4 2019-12-09T11:39:02.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.354+0000 7ff56f1b2ec0 20 remove_watcher() i=5 2019-12-09T11:39:02.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.416+0000 7ff56f1b2ec0 20 remove_watcher() i=6 2019-12-09T11:39:02.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T11:39:02.457+0000 7ff56f1b2ec0 20 remove_watcher() i=7 2019-12-09T11:39:02.461 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-12-09T11:39:03.087 INFO:teuthology.orchestra.run.smithi174.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-12-09T11:39:06.387 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-12-09T11:39:06.586 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-12-09T11:39:07.491 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :clean UP-TO-DATE 2019-12-09T11:39:07.587 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-12-09T11:39:07.787 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-12-09T11:39:07.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-12-09T11:39:07.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-12-09T11:39:07.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-12-09T11:39:07.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-12-09T11:39:07.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-12-09T11:39:07.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-12-09T11:39:07.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-12-09T11:39:08.087 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-12-09T11:39:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-12-09T11:39:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-12-09T11:39:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-12-09T11:39:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-12-09T11:39:08.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-12-09T11:39:08.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-12-09T11:39:08.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-12-09T11:39:08.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-12-09T11:39:08.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-12-09T11:39:08.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-12-09T11:39:08.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-12-09T11:39:08.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-12-09T11:39:08.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-12-09T11:39:08.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-12-09T11:39:08.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-12-09T11:39:08.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-12-09T11:39:08.294 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-12-09T11:39:08.294 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-12-09T11:39:08.294 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-12-09T11:39:08.294 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-12-09T11:39:08.294 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-12-09T11:39:08.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-12-09T11:39:08.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-12-09T11:39:08.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-12-09T11:39:08.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-12-09T11:39:08.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-12-09T11:39:08.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-12-09T11:39:08.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-12-09T11:39:08.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-12-09T11:39:08.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-12-09T11:39:08.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-12-09T11:39:08.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-12-09T11:39:08.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-12-09T11:39:08.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-12-09T11:39:08.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-12-09T11:39:08.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-12-09T11:39:08.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-12-09T11:39:08.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-12-09T11:39:08.498 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-12-09T11:39:08.498 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-12-09T11:39:08.498 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-12-09T11:39:08.498 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-12-09T11:39:08.499 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-12-09T11:39:08.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-12-09T11:39:08.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-12-09T11:39:08.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-12-09T11:39:08.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-12-09T11:39:08.591 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-12-09T11:39:08.592 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-12-09T11:39:08.593 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-12-09T11:39:08.594 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-12-09T11:39:08.594 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-12-09T11:39:08.594 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-12-09T11:39:08.692 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-12-09T11:39:08.693 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-12-09T11:39:08.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-12-09T11:39:08.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-12-09T11:39:08.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-12-09T11:39:08.697 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-12-09T11:39:08.697 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-12-09T11:39:08.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-12-09T11:39:08.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-12-09T11:39:08.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-12-09T11:39:08.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-12-09T11:39:08.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-12-09T11:39:08.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-12-09T11:39:08.790 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-12-09T11:39:08.791 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-12-09T11:39:08.791 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-12-09T11:39:08.791 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-12-09T11:39:08.791 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-12-09T11:39:08.887 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-12-09T11:39:08.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-12-09T11:39:08.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-12-09T11:39:08.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-12-09T11:39:08.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-12-09T11:39:08.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-12-09T11:39:08.889 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-12-09T11:39:08.889 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-12-09T11:39:08.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-12-09T11:39:08.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-12-09T11:39:09.087 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-12-09T11:39:09.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-12-09T11:39:09.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-12-09T11:39:09.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-12-09T11:39:09.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-12-09T11:39:09.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-12-09T11:39:09.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-12-09T11:39:09.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-12-09T11:39:09.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-12-09T11:39:09.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-12-09T11:39:09.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-12-09T11:39:09.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-12-09T11:39:09.287 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-12-09T11:39:09.287 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-12-09T11:39:09.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-12-09T11:39:09.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-12-09T11:39:09.387 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-12-09T11:39:09.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-12-09T11:39:09.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-12-09T11:39:09.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-12-09T11:39:10.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-12-09T11:39:10.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-12-09T11:39:10.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-12-09T11:39:10.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-12-09T11:39:10.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-12-09T11:39:10.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-12-09T11:39:10.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-12-09T11:39:10.291 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-12-09T11:39:10.292 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-12-09T11:39:10.293 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-12-09T11:39:10.387 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-12-09T11:39:10.387 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-12-09T11:39:10.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-12-09T11:39:10.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-12-09T11:39:10.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-12-09T11:39:10.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-12-09T11:39:10.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-12-09T11:39:10.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-12-09T11:39:10.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-12-09T11:39:10.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-12-09T11:39:10.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-12-09T11:39:10.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-12-09T11:39:10.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-12-09T11:39:10.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-12-09T11:39:10.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-12-09T11:39:10.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-12-09T11:39:10.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-12-09T11:39:10.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-12-09T11:39:10.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-12-09T11:39:10.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-12-09T11:39:10.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-12-09T11:39:10.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-12-09T11:39:10.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-12-09T11:39:10.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-12-09T11:39:10.394 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-12-09T11:39:10.394 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-12-09T11:39:10.395 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-12-09T11:39:10.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-12-09T11:39:10.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-12-09T11:39:10.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-12-09T11:39:10.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-12-09T11:39:10.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-12-09T11:39:10.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-12-09T11:39:10.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-12-09T11:39:10.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-12-09T11:39:10.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-12-09T11:39:10.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-12-09T11:39:10.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-12-09T11:39:10.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-12-09T11:39:10.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-12-09T11:39:10.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-12-09T11:39:10.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-12-09T11:39:10.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-12-09T11:39:10.491 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-12-09T11:39:10.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-12-09T11:39:10.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-12-09T11:39:10.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-12-09T11:39:10.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-12-09T11:39:10.492 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-12-09T11:39:10.587 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-12-09T11:39:10.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-12-09T11:39:10.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-12-09T11:39:10.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-12-09T11:39:10.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-12-09T11:39:10.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-12-09T11:39:10.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-12-09T11:39:10.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-12-09T11:39:10.694 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-12-09T11:39:10.695 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-12-09T11:39:10.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-12-09T11:39:10.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-12-09T11:39:10.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-12-09T11:39:10.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-12-09T11:39:10.696 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-12-09T11:39:10.787 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-12-09T11:39:10.787 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-12-09T11:39:10.788 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-12-09T11:39:10.789 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-12-09T11:39:10.887 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-12-09T11:39:10.887 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-12-09T11:39:10.887 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-12-09T11:39:10.888 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-12-09T11:39:10.889 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-12-09T11:39:10.889 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-12-09T11:39:10.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-12-09T11:39:10.987 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-12-09T11:39:10.988 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-12-09T11:39:10.989 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-12-09T11:39:10.989 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-12-09T11:39:10.989 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-12-09T11:39:11.088 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-12-09T11:39:11.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-12-09T11:39:11.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-12-09T11:39:11.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-12-09T11:39:11.089 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-12-09T11:39:11.187 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-12-09T11:39:11.188 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-12-09T11:39:11.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-12-09T11:39:11.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-12-09T11:39:11.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-12-09T11:39:11.189 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-12-09T11:39:11.287 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-12-09T11:39:11.287 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-12-09T11:39:11.288 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-12-09T11:39:11.289 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-12-09T11:39:11.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-12-09T11:39:11.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-12-09T11:39:11.290 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-12-09T11:39:11.388 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-12-09T11:39:11.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-12-09T11:39:11.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-12-09T11:39:11.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-12-09T11:39:11.389 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-12-09T11:39:11.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-12-09T11:39:11.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-12-09T11:39:11.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-12-09T11:39:11.390 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-12-09T11:39:11.391 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-12-09T11:39:11.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-12-09T11:39:11.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-12-09T11:39:11.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-12-09T11:39:11.392 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-12-09T11:39:11.393 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-12-09T11:39:11.393 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-12-09T11:39:11.393 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-12-09T11:39:11.393 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-12-09T11:39:11.393 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-12-09T11:39:11.394 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-12-09T11:39:11.394 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-12-09T11:39:11.487 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-12-09T11:39:11.488 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-12-09T11:39:11.489 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-12-09T11:39:11.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-12-09T11:39:11.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-12-09T11:39:11.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-12-09T11:39:11.490 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-12-09T11:39:11.587 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-12-09T11:39:11.587 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-12-09T11:39:11.588 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-12-09T11:39:11.589 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-12-09T11:39:11.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-12-09T11:39:11.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-12-09T11:39:11.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-12-09T11:39:11.590 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-12-09T11:39:11.687 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-12-09T11:39:11.687 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-12-09T11:39:11.688 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-12-09T11:39:11.689 INFO:teuthology.orchestra.run.smithi174.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-12-09T11:39:20.388 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileJava 2019-12-09T11:39:20.388 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processResources 2019-12-09T11:39:20.388 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :classes 2019-12-09T11:39:20.788 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:39:20.788 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileTestJava 2019-12-09T11:39:20.788 INFO:teuthology.orchestra.run.smithi174.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T11:39:20.788 INFO:teuthology.orchestra.run.smithi174.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T11:39:20.788 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:39:20.789 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T11:39:20.789 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :testClasses 2019-12-09T11:39:28.687 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:39:28.688 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :test 2019-12-09T11:39:28.688 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:39:28.688 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-12-09T11:40:37.889 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:40:37.890 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-12-09T11:40:43.189 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:40:43.189 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-12-09T11:40:45.189 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:40:45.189 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-12-09T11:40:51.689 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:40:51.690 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-12-09T11:41:47.591 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:41:47.591 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-12-09T11:41:55.891 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:41:55.892 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T11:41:56.291 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:41:56.291 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T11:41:57.491 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:41:57.491 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-12-09T11:41:59.491 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:41:59.491 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-12-09T11:42:06.391 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:42:06.391 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-12-09T11:42:53.362 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:21:45.211 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.381 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:21:45.231 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.393 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:45.222 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.409 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:45.238 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.410 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:45.239 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.410 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:45.240 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.523 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:21:45.372 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.629 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:21:45.458 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:42:53.633 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:21:45.462 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:43:02.692 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:43:02.692 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-12-09T11:43:17.992 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:43:17.993 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-12-09T11:46:54.272 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:25:46.122 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.299 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:25:46.127 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.330 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:46.156 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.332 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:46.159 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.338 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:46.165 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.342 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:46.168 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.374 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:25:46.224 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.464 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:25:46.290 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:46:54.468 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:25:46.294 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:47:14.895 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:47:14.895 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-12-09T11:47:32.995 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:47:32.995 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-12-09T11:47:35.595 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:47:35.595 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-12-09T11:47:42.395 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:47:42.395 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-12-09T11:47:47.495 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:47:47.495 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-12-09T11:48:00.695 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:48:00.698 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-12-09T11:48:49.295 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:48:49.296 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-12-09T11:48:58.195 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:48:58.195 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-12-09T11:48:58.695 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:48:58.695 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-12-09T11:48:59.795 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:48:59.795 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-12-09T11:49:58.359 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T11:49:58.303+0000 11e433700 -1 handle_data failed with exit code -104 2019-12-09T11:49:59.395 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:49:59.395 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-12-09T11:50:05.995 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T11:50:05.996 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T11:53:18.202 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:32:10.052 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.221 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:10.044 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.222 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:32:10.073 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.223 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:10.048 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.225 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:10.050 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.230 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:10.055 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.440 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:32:10.265 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.449 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:32:10.300 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:53:18.465 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:32:10.290 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.406 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:37:21.258 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.441 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:21.256 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.441 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:21.263 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.441 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:21.257 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.441 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:21.263 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.441 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:37:21.268 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.574 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:37:21.425 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.656 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:37:21.481 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T11:58:29.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:37:21.484 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:38.995 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:30.846 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.018 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:30.832 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.018 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:30.835 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.018 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:30.834 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.019 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:30.840 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.019 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:30.851 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.168 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:31.019 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.225 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:31.050 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:39.242 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:31.066 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.920 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:34.771 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.924 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:34.776 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.942 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:42:34.793 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.962 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:34.787 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.963 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:34.788 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.967 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:34.792 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.970 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:34.794 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.980 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:42:34.805 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:42.982 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:42:34.806 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:03:57.296 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:03:57.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T12:05:03.596 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:05:03.597 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-12-09T12:06:03.599 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:06:03.612 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-12-09T12:06:14.597 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:06:14.597 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-12-09T12:08:52.785 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:47:44.636 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.788 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:47:44.612 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.791 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:47:44.642 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.791 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:47:44.615 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.800 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:47:44.624 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.803 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:47:44.628 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.977 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:47:44.801 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:52.979 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:47:44.803 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:08:53.017 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:47:44.868 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:04.967 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:50:56.818 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:04.989 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:50:56.828 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.006 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:50:56.830 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.038 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:50:56.862 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.059 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:50:56.883 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.067 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:50:56.891 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.088 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:50:56.939 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.131 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:50:56.944 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:12:05.218 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:50:57.042 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:04.982 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:52:56.833 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:04.986 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:52:56.837 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.002 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:52:56.826 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.008 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:52:56.832 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.047 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:52:56.871 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.054 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:52:56.878 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.148 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:52:56.972 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.153 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:52:57.004 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:14:05.208 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:52:57.032 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.783 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:58:09.634 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.795 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:58:09.646 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.814 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:58:09.638 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.815 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:58:09.640 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.820 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:58:09.644 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.821 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:58:09.646 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:17.990 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:58:09.813 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:18.045 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:58:09.896 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:19:18.079 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:58:09.903 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:10.397 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:20:10.397 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-12-09T12:20:19.697 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:20:19.697 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-12-09T12:20:27.057 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:59:18.908 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.060 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:59:18.911 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.061 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:59:18.885 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.068 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:59:18.891 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.074 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:59:18.898 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.078 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:59:18.902 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.115 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:00:59:18.938 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.130 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:00:59:18.954 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:20:27.132 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:00:59:18.983 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:21:53.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:21:53.198 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-12-09T12:22:02.297 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:22:02.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-12-09T12:23:06.297 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:23:06.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-12-09T12:23:13.397 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:23:13.397 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-12-09T12:23:26.997 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:23:26.997 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-12-09T12:23:48.397 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:23:48.397 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-12-09T12:23:50.400 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:23:50.400 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-12-09T12:24:04.297 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:04.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-12-09T12:24:09.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:09.197 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-12-09T12:24:22.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:22.197 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-12-09T12:24:27.697 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:27.697 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-12-09T12:24:28.811 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:03:20.661 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.819 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:03:20.670 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.825 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:03:20.649 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.828 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:03:20.652 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.828 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:03:20.652 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:03:20.656 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.975 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:03:20.826 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:28.985 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:03:20.809 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:29.017 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:03:20.841 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:24:45.297 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:45.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-12-09T12:24:51.297 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:24:51.297 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-12-09T12:25:26.697 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:25:26.697 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-12-09T12:25:41.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:25:41.197 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-12-09T12:26:34.097 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:26:34.099 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-12-09T12:26:49.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:26:49.197 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-12-09T12:27:53.198 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:27:53.199 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-12-09T12:28:08.997 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:28:08.997 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-12-09T12:28:48.894 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:07:40.745 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:48.900 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:07:40.752 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:48.911 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:07:40.735 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:48.923 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:07:40.747 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:48.939 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:07:40.763 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:48.942 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:07:40.767 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:49.007 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:07:40.858 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:49.066 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:07:40.890 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:28:49.081 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:07:40.905 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.228 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:08:32.080 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.235 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:08:32.086 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.237 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:08:32.062 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.242 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:08:32.067 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.245 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:08:32.069 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.250 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:08:32.075 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.294 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:08:32.145 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.312 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:08:32.135 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:40.313 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:08:32.137 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:29:54.597 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:29:54.597 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-12-09T12:31:36.197 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:31:36.199 INFO:teuthology.orchestra.run.smithi174.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-12-09T12:33:50.097 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:33:50.097 INFO:teuthology.orchestra.run.smithi174.stdout:---------------------------------------------------------------------- 2019-12-09T12:33:50.097 INFO:teuthology.orchestra.run.smithi174.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-12-09T12:33:50.097 INFO:teuthology.orchestra.run.smithi174.stdout:---------------------------------------------------------------------- 2019-12-09T12:33:50.349 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:33:50.350 INFO:teuthology.orchestra.run.smithi174.stdout:BUILD SUCCESSFUL in 54m 47s 2019-12-09T12:33:50.350 INFO:teuthology.orchestra.run.smithi174.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-12-09T12:33:50.666 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:33:50.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.825+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:50.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.857+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:50.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.858+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:50.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.870+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:50.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.870+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:50.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.879+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:50.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.879+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:50.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:50.897+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:33:51.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.076+0000 7f125037dec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:33:51.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.076+0000 7f125037dec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:33:51.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.076+0000 7f125037dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:33:51.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.076+0000 7f125037dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:33:51.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.076+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.090 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.088+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:33:51.091 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.088+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.106 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.103+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:33:51.106 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.103+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.111+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:33:51.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.111+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.119+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:33:51.121 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.119+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.125+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:51.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.125+0000 7f125037dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:33:51.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.125+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.133 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.131+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:51.134 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.131+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.143 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.140+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:33:51.143 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.140+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.150 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.148+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:33:51.151 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.148+0000 7f125037dec0 20 zonegroup default 2019-12-09T12:33:51.151 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.148+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.153+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:51.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.153+0000 7f125037dec0 10 Cannot find current period zone using local zone 2019-12-09T12:33:51.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.153+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.161 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.158+0000 7f125037dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:33:51.161 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.158+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.165+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:33:51.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.165+0000 7f125037dec0 20 rados->read ofs=0 len=0 2019-12-09T12:33:51.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.175+0000 7f125037dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:33:51.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.175+0000 7f125037dec0 20 zone default found 2019-12-09T12:33:51.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.175+0000 7f125037dec0 20 started sync module instance, tier type = 2019-12-09T12:33:51.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.175+0000 7f125037dec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:33:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=0 2019-12-09T12:33:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=1 2019-12-09T12:33:51.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=2 2019-12-09T12:33:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=3 2019-12-09T12:33:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=4 2019-12-09T12:33:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=5 2019-12-09T12:33:51.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.521+0000 7f125037dec0 20 add_watcher() i=6 2019-12-09T12:33:51.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.557+0000 7f125037dec0 20 add_watcher() i=7 2019-12-09T12:33:51.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.557+0000 7f125037dec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:33:51.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.557+0000 7f123d5a7700 0 -- 172.21.15.174:0/3448459597 >> [v2:172.21.15.174:6802/10741,v1:172.21.15.174:6803/10741] conn(0x7f1234020550 msgr2=0x7f1234044af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:33:51.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.557+0000 7f120dfdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:33:51.611 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:51.608+0000 7f123d5a7700 0 -- 172.21.15.174:0/423460708 >> v2:172.21.15.174:3300/0 conn(0x5560582caea0 msgr2=0x55605830c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:33:52.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.893+0000 7f117ffff700 20 reqs_thread_entry: start 2019-12-09T12:33:52.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.894+0000 7f117dffb700 20 reqs_thread_entry: start 2019-12-09T12:33:52.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.894+0000 7f125037dec0 20 init_complete bucket index max shards: 0 2019-12-09T12:33:52.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.894+0000 7f116bfff700 20 reqs_thread_entry: start 2019-12-09T12:33:52.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.894+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:33:52.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.953+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:52.978 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.975+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:52.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.454', time=2019-12-09T13:46:01.795239+0000, chain.objs.size()=9 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.1 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.1_1 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.2 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.2_1 2019-12-09T12:33:52.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.3 2019-12-09T12:33:52.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.3_1 2019-12-09T12:33:52.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.4 2019-12-09T12:33:52.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.4_1 2019-12-09T12:33:52.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:52.989+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~BEz_ftB3Zmzho0BZj3vzd68U78mAwWC.5 2019-12-09T12:33:53.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.285+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:53.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.409+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:33:53.457 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.453+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:53.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.471+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:53.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:53.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.466', time=2019-12-09T13:47:13.257838+0000, chain.objs.size()=7 2019-12-09T12:33:53.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.1 2019-12-09T12:33:53.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.1_1 2019-12-09T12:33:53.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.1_2 2019-12-09T12:33:53.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.2 2019-12-09T12:33:53.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.2_1 2019-12-09T12:33:53.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.2_2 2019-12-09T12:33:53.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.484+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~8-IFjkBulWS0I-kUd8lXqpgW0UQQei6.3 2019-12-09T12:33:53.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.678+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:53.848 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.845+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:33:53.887 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.884+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:53.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.898+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:53.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:53.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.417', time=2019-12-09T13:43:11.179006+0000, chain.objs.size()=9 2019-12-09T12:33:53.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.1 2019-12-09T12:33:53.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.1_1 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.2 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.2_1 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.918+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.3 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.919+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.3_1 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.919+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.4 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.919+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.4_1 2019-12-09T12:33:53.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:53.919+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.5 2019-12-09T12:33:54.093 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.091+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:54.179 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.176+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:33:54.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.218+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:54.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.233+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:54.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.250+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T12:33:54.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.250+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6', time=2019-12-09T13:40:37.430459+0000, chain.objs.size()=9 2019-12-09T12:33:54.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.250+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__multipart_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.1 2019-12-09T12:33:54.254 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.250+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__shadow_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.1_1 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.250+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__multipart_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.2 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__shadow_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.2_1 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__multipart_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.3 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__shadow_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.3_1 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__multipart_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.4 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__shadow_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.4_1 2019-12-09T12:33:54.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.13__multipart_key1.2~coZMcwtSgH3nG_PaAdG9xO9tFyvr0y6.5 2019-12-09T12:33:54.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.430', time=2019-12-09T13:47:17.929047+0000, chain.objs.size()=5 2019-12-09T12:33:54.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.19__shadow_.1sWv0DI8G2DzHuHdqWUjHGRu6DqzGfq_1 2019-12-09T12:33:54.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.19__shadow_.1sWv0DI8G2DzHuHdqWUjHGRu6DqzGfq_2 2019-12-09T12:33:54.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.251+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.19__shadow_.1sWv0DI8G2DzHuHdqWUjHGRu6DqzGfq_3 2019-12-09T12:33:54.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.19__shadow_.1sWv0DI8G2DzHuHdqWUjHGRu6DqzGfq_4 2019-12-09T12:33:54.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.19__shadow_.1sWv0DI8G2DzHuHdqWUjHGRu6DqzGfq_5 2019-12-09T12:33:54.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.755', time=2019-12-09T14:21:56.192460+0000, chain.objs.size()=9 2019-12-09T12:33:54.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__multipart_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.1 2019-12-09T12:33:54.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__shadow_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.1_1 2019-12-09T12:33:54.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__multipart_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.2 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__shadow_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.2_1 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__multipart_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.3 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__shadow_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.3_1 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__multipart_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.4 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__shadow_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.4_1 2019-12-09T12:33:54.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:54.582+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.29__multipart_key1.2~ihwQMqB6iPlOUn9NF8AS9J8_PmrD7AX.5 2019-12-09T12:33:55.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.033+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:55.120 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.117+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:33:55.171 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.168+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:55.188 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.185+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:55.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.704', time=2019-12-09T14:14:22.588271+0000, chain.objs.size()=21 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.1 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.1_1 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.2 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.2_1 2019-12-09T12:33:55.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.3 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.3_1 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.4 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.4_1 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.5 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.5_1 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.6 2019-12-09T12:33:55.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.6_1 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.7 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.7_1 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.8 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.8_1 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.9 2019-12-09T12:33:55.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.9_1 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.10 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.10_1 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~5rqbvuoLx20UljmpOez2yPeEPa65HNN.11 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0', time=2019-12-09T14:23:41.504958+0000, chain.objs.size()=26 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.1 2019-12-09T12:33:55.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.522+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.2 2019-12-09T12:33:55.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.803+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.3 2019-12-09T12:33:55.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.803+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.4 2019-12-09T12:33:55.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.803+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.5 2019-12-09T12:33:55.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.803+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.6 2019-12-09T12:33:55.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.803+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.7 2019-12-09T12:33:55.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.8 2019-12-09T12:33:55.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.9 2019-12-09T12:33:55.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.10 2019-12-09T12:33:55.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.11 2019-12-09T12:33:55.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.12 2019-12-09T12:33:55.911 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:55.907+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.13 2019-12-09T12:33:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.107+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.rSeNGKOmz9S8MJjoA56IAEvo9qlXA9q.14 2019-12-09T12:33:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.107+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.15 2019-12-09T12:33:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.107+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.aMxJauXkCpyEhCzPpi4cftjZW2cU3TX.16 2019-12-09T12:33:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.107+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.ZFSWiSKjfUaeyOWmxktrmHpjRKteifz.17 2019-12-09T12:33:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.107+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.fV5goxju5BsJdn_X02VA2BkOI2l-XZW.18 2019-12-09T12:33:56.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.ALPzRT7T179cjH4ihZcA2FR2-wOihmA.19 2019-12-09T12:33:56.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.7NKRRg490nemr9rRhHxATGYAFqJHG9o.20 2019-12-09T12:33:56.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.21 2019-12-09T12:33:56.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.22 2019-12-09T12:33:56.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.23 2019-12-09T12:33:56.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.197+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.24 2019-12-09T12:33:56.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.241+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.25 2019-12-09T12:33:56.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.241+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.31__multipart_key1.2~11kIabdG9hUj8LZFsxCpuh3sA7fbCm0.26 2019-12-09T12:33:56.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.501+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:56.612 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.609+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:33:56.648 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.645+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:56.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.660+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:56.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:56.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.716', time=2019-12-09T14:16:20.295219+0000, chain.objs.size()=19 2019-12-09T12:33:56.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.1 2019-12-09T12:33:56.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.1_1 2019-12-09T12:33:56.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.2 2019-12-09T12:33:56.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.2_1 2019-12-09T12:33:56.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.3 2019-12-09T12:33:56.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.3_1 2019-12-09T12:33:56.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.4 2019-12-09T12:33:56.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.4_1 2019-12-09T12:33:56.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.5 2019-12-09T12:33:56.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.5_1 2019-12-09T12:33:56.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.6 2019-12-09T12:33:56.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.6_1 2019-12-09T12:33:56.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.851+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.7 2019-12-09T12:33:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.895+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.7_1 2019-12-09T12:33:56.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:56.895+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.8 2019-12-09T12:33:57.050 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.047+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.8_1 2019-12-09T12:33:57.050 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.047+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.9 2019-12-09T12:33:57.050 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.047+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.9_1 2019-12-09T12:33:57.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.047+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~n0wt2SlcKRS12U71w_jEWAdoHHeRmUf.10 2019-12-09T12:33:57.273 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.270+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:57.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.377+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:33:57.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.420+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:57.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.453+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:57.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:57.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.460', time=2019-12-09T13:46:42.622925+0000, chain.objs.size()=7 2019-12-09T12:33:57.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.1 2019-12-09T12:33:57.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.1_1 2019-12-09T12:33:57.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.1_2 2019-12-09T12:33:57.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.2 2019-12-09T12:33:57.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.2_1 2019-12-09T12:33:57.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.2_2 2019-12-09T12:33:57.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.474+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fA-KGkRQBMLVY8bVGJv-1RoX_-pAU2V.3 2019-12-09T12:33:57.679 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.676+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:57.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.839+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:33:57.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.896+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:57.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.931+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:57.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.415', time=2019-12-09T13:43:05.074343+0000, chain.objs.size()=9 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.1 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.1_1 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.2 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.2_1 2019-12-09T12:33:57.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.3 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.3_1 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.4 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__shadow_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.4_1 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.17__multipart_key1.2~Ntovj4nVsw8O_iy0gpBTd0-e_4m9NLQ.5 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.472', time=2019-12-09T13:47:24.331770+0000, chain.objs.size()=7 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.1 2019-12-09T12:33:57.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.1_1 2019-12-09T12:33:57.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:57.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.1_2 2019-12-09T12:33:58.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.099+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.2 2019-12-09T12:33:58.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.128+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.2_1 2019-12-09T12:33:58.131 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.128+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.2_2 2019-12-09T12:33:58.241 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.238+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fTd3QnT7ymQ3zhbDLbi9nFB0YyzEqkB.3 2019-12-09T12:33:58.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.458+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:58.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.572+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:33:58.618 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.615+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:58.638 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.636+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:58.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.647+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:58.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.716+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:33:58.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.760+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:58.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.775+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:58.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:58.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.386', time=2019-12-09T13:41:49.870095+0000, chain.objs.size()=9 2019-12-09T12:33:58.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__multipart_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.1 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__shadow_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.1_1 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__multipart_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.2 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__shadow_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.2_1 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__multipart_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.3 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__shadow_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.3_1 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__multipart_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.4 2019-12-09T12:33:58.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__shadow_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.4_1 2019-12-09T12:33:58.802 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:58.797+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.15__multipart_key1.2~edVYrBuD1Myg4cpVGLKveLAqdUY0lz_.5 2019-12-09T12:33:59.054 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.051+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:33:59.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.146+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:33:59.194 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.191+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.206+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:59.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.214+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:59.253 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.250+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:33:59.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.302+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:33:59.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.317+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:33:59.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.724', time=2019-12-09T14:18:16.749475+0000, chain.objs.size()=16 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.1 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.1_1 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.1_2 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.2 2019-12-09T12:33:59.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.2_1 2019-12-09T12:33:59.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.2_2 2019-12-09T12:33:59.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.3 2019-12-09T12:33:59.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.3_1 2019-12-09T12:33:59.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.3_2 2019-12-09T12:33:59.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.4 2019-12-09T12:33:59.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.4_1 2019-12-09T12:33:59.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.332+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.4_2 2019-12-09T12:33:59.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.577+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.5 2019-12-09T12:33:59.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.577+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.5_1 2019-12-09T12:33:59.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.624+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.5_2 2019-12-09T12:33:59.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.624+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~sDNoPESBwWQobDVRUy2sYy8UuyiK5CJ.6 2019-12-09T12:33:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:33:59.903+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:00.028 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.025+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:34:00.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.081+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.109 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.106+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.116+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.154 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.151+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:34:00.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.215+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.236 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.234+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.245 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.242+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.275+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:34:00.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.318+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.333+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.343+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.395+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:34:00.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.446+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.468+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.476+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.512+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:34:00.577 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.572+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.596+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.606+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.672 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.669+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:34:00.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.718+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:00.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.733+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:00.759 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:34:00.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~Ki46sYCaBsliXWxzvLm5wKyJZ_sWNQ3', time=2019-12-09T14:04:48.909598+0000, chain.objs.size()=9 2019-12-09T12:34:00.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file.mpg.iGTUIBjIbedItncghZtVWszA2fYC_uo.1 2019-12-09T12:34:00.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file.mpg.iGTUIBjIbedItncghZtVWszA2fYC_uo.1_1 2019-12-09T12:34:00.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file.mpg.335P7LIBc7MhYer79ad-IWJWrXKRd8b.2 2019-12-09T12:34:00.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file.mpg.335P7LIBc7MhYer79ad-IWJWrXKRd8b.2_1 2019-12-09T12:34:00.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file.mpg.QPsGrICla5XtKR_DL4GRwx6AOX4wDF7.3 2019-12-09T12:34:00.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file.mpg.QPsGrICla5XtKR_DL4GRwx6AOX4wDF7.3_1 2019-12-09T12:34:00.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.756+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file.mpg.I07Vkes4GA7b3CG8O8jRBDJ0tNnZpnc.4 2019-12-09T12:34:00.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file.mpg.I07Vkes4GA7b3CG8O8jRBDJ0tNnZpnc.4_1 2019-12-09T12:34:00.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file.mpg.xGzha6rYNjX4I-HptpI_1suvfoP094T.5 2019-12-09T12:34:00.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.732', time=2019-12-09T14:20:09.022500+0000, chain.objs.size()=16 2019-12-09T12:34:00.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.1 2019-12-09T12:34:00.762 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.1_1 2019-12-09T12:34:00.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.757+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.1_2 2019-12-09T12:34:00.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.927+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.2 2019-12-09T12:34:00.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.942+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.2_1 2019-12-09T12:34:00.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.955+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.2_2 2019-12-09T12:34:00.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:00.955+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.3 2019-12-09T12:34:01.016 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.014+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.3_1 2019-12-09T12:34:01.017 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.014+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.3_2 2019-12-09T12:34:01.017 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.014+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.4 2019-12-09T12:34:01.017 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.014+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.4_1 2019-12-09T12:34:01.017 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.014+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.4_2 2019-12-09T12:34:01.022 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.020+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.5 2019-12-09T12:34:01.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.241+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.5_1 2019-12-09T12:34:01.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.241+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.5_2 2019-12-09T12:34:01.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.241+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~RH8TYsJKMmbE2f_1WCnQIHi2KsOj2iK.6 2019-12-09T12:34:01.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.733+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:01.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.860+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:34:01.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.904+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:01.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.919+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:01.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:34:01.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.528', time=2019-12-09T13:50:01.372553+0000, chain.objs.size()=9 2019-12-09T12:34:01.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__multipart_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.1 2019-12-09T12:34:01.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__shadow_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.1_1 2019-12-09T12:34:01.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__multipart_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.2 2019-12-09T12:34:01.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__shadow_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.2_1 2019-12-09T12:34:01.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__multipart_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.3 2019-12-09T12:34:01.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__shadow_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.3_1 2019-12-09T12:34:01.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__multipart_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.4 2019-12-09T12:34:01.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__shadow_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.4_1 2019-12-09T12:34:01.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:01.932+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.25__multipart_key1.2~HlCnvbaJsRPPIsW8wyDMOQofvbbcLhL.5 2019-12-09T12:34:02.188 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.185+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:02.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.274+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:34:02.315 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.313+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.328+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:02.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.336+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.377 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.374+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:34:02.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.424+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.443 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.440+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:02.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.453+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.489+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:34:02.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.533+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.548+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:02.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.559+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.598+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:34:02.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.653+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:02.675 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.672+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:34:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~i8ghkjcOTZ_nPS_TvSLlQcHWn6eDJr0', time=2019-12-09T14:04:50.387835+0000, chain.objs.size()=9 2019-12-09T12:34:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file2.mpg.iHMR0GR-aRJ_EQGRLtkoPosdSFLpo00.1 2019-12-09T12:34:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file2.mpg.iHMR0GR-aRJ_EQGRLtkoPosdSFLpo00.1_1 2019-12-09T12:34:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file2.mpg.pbJBF3alL2iBmrlqBzX1any8jtrsS5z.2 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file2.mpg.pbJBF3alL2iBmrlqBzX1any8jtrsS5z.2_1 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file2.mpg._F2LLo6DqU2KaMYVDHv7JkEK5hYVgrb.3 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file2.mpg._F2LLo6DqU2KaMYVDHv7JkEK5hYVgrb.3_1 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file2.mpg.klpYJguFTrc8lafZzq0n6zX8Hul3PeM.4 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.687+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__shadow_file2.mpg.klpYJguFTrc8lafZzq0n6zX8Hul3PeM.4_1 2019-12-09T12:34:02.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.688+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.26__multipart_file2.mpg.2~i8ghkjcOTZ_nPS_TvSLlQcHWn6eDJr0.5 2019-12-09T12:34:02.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:02.925+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:03.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.017+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:34:03.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.069+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:03.088 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.085+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:03.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:34:03.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.691', time=2019-12-09T14:11:32.322855+0000, chain.objs.size()=21 2019-12-09T12:34:03.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.1 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.1_1 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.2 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.2_1 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.3 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.3_1 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.4 2019-12-09T12:34:03.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.4_1 2019-12-09T12:34:03.117 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.5 2019-12-09T12:34:03.117 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.5_1 2019-12-09T12:34:03.117 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.6 2019-12-09T12:34:03.117 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.112+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.6_1 2019-12-09T12:34:03.280 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.278+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.7 2019-12-09T12:34:03.292 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.290+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.7_1 2019-12-09T12:34:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.8 2019-12-09T12:34:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.8_1 2019-12-09T12:34:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.9 2019-12-09T12:34:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.9_1 2019-12-09T12:34:03.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.10 2019-12-09T12:34:03.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.10_1 2019-12-09T12:34:03.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~TyfofcWLQ7AYkAprB-2M7S_-WXxxYCl.11 2019-12-09T12:34:03.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.740', time=2019-12-09T14:20:13.713349+0000, chain.objs.size()=16 2019-12-09T12:34:03.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.411+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.1 2019-12-09T12:34:03.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.457+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.1_1 2019-12-09T12:34:03.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.457+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.1_2 2019-12-09T12:34:03.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.457+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.2 2019-12-09T12:34:03.675 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.672+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.2_1 2019-12-09T12:34:03.687 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.684+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.2_2 2019-12-09T12:34:03.798 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.795+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.3 2019-12-09T12:34:03.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.3_1 2019-12-09T12:34:03.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.3_2 2019-12-09T12:34:03.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.4 2019-12-09T12:34:03.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.4_1 2019-12-09T12:34:03.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.4_2 2019-12-09T12:34:03.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.5 2019-12-09T12:34:03.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.5_1 2019-12-09T12:34:03.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__shadow_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.5_2 2019-12-09T12:34:03.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:03.865+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.28__multipart_key1.2~yS-HMEdNPy6AIzNnn1MGqY4l4R55RPI.6 2019-12-09T12:34:04.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.213+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:04.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.337+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:34:04.380 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.377+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:04.400 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.398+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:04.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.407+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:04.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.466+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:34:04.517 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.514+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:04.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.529+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.539+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T12:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.438', time=2019-12-09T13:44:59.440326+0000, chain.objs.size()=9 2019-12-09T12:34:04.543 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.1 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.1_1 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.2 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.2_1 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.3 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.3_1 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.4 2019-12-09T12:34:04.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.4_1 2019-12-09T12:34:04.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.540+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~fwnOaJUdndNspP1hU1JQI8vtzcv1coz.5 2019-12-09T12:34:04.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.763+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:04.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.876+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:34:04.924 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.921+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:04.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.936+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:04.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T12:34:04.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.446', time=2019-12-09T13:45:32.217362+0000, chain.objs.size()=9 2019-12-09T12:34:04.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.1 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.1_1 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.2 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.2_1 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.3 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.3_1 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.4 2019-12-09T12:34:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__shadow_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.4_1 2019-12-09T12:34:04.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.20__multipart_key1.2~KilbnFaidMpmuhkU53L_FzFluFk6jmb.5 2019-12-09T12:34:04.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.770', time=2019-12-09T14:23:08.797247+0000, chain.objs.size()=9 2019-12-09T12:34:04.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__multipart_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.1 2019-12-09T12:34:04.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__shadow_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.1_1 2019-12-09T12:34:04.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:04.953+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__multipart_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.2 2019-12-09T12:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.196+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__shadow_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.2_1 2019-12-09T12:34:05.199 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.196+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__multipart_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.3 2019-12-09T12:34:05.358 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.355+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__shadow_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.3_1 2019-12-09T12:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.355+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__multipart_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.4 2019-12-09T12:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.355+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__shadow_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.4_1 2019-12-09T12:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.355+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.30__multipart_key1.2~mCszEmjLDciORK1I-rowXwrL-ktqTtM.5 2019-12-09T12:34:05.379 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.355+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~mGBo0BWeM7C0lAxo0PTKYR2VzmxoV0e', time=2019-12-09T14:33:29.991583+0000, chain.objs.size()=0 2019-12-09T12:34:05.560 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.558+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:05.657 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.654+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:34:05.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.695+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:05.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.712+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T12:34:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.509', time=2019-12-09T13:48:53.271437+0000, chain.objs.size()=9 2019-12-09T12:34:05.731 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__multipart_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.1 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__shadow_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.1_1 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__multipart_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.2 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__shadow_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.2_1 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__multipart_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.3 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__shadow_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.3_1 2019-12-09T12:34:05.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__multipart_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.4 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__shadow_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.4_1 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.24__multipart_key1.2~0Ck0ESUr-4cZ3Yeo6Cs-6zcRbGk9E3r.5 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 20 garbage collection: RGWGC::process iterating over entry tag='2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo', time=2019-12-09T14:06:07.649414+0000, chain.objs.size()=5 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.27__multipart_key1.2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo.7 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.27__shadow_key1.2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo.7_1 2019-12-09T12:34:05.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.728+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.27__multipart_key1.2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo.10 2019-12-09T12:34:05.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:05.855+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.27__shadow_key1.2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo.10_1 2019-12-09T12:34:06.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.073+0000 7f125037dec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.27__multipart_key1.2~LMeK1QXao-q3ZfLHgqzWaIZwqgLptPo.13 2019-12-09T12:34:06.231 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.228+0000 7f125037dec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T12:34:06.321 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.318+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:34:06.373 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.370+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.385+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:06.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.394+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.430 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.427+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:34:06.470 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.467+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.483+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:06.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.491+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.520 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.517+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:34:06.594 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.591+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.624 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.621+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:06.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.654+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.698+0000 7f125037dec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:34:06.802 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.799+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.849 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.846+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:06.867 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.864+0000 7f125037dec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:06.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.965+0000 7f125037dec0 20 remove_watcher() i=0 2019-12-09T12:34:06.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:06.965+0000 7f125037dec0 2 removed watcher, disabling cache 2019-12-09T12:34:07.007 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.004+0000 7f125037dec0 20 remove_watcher() i=1 2019-12-09T12:34:07.042 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.039+0000 7f125037dec0 20 remove_watcher() i=2 2019-12-09T12:34:07.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.069+0000 7f125037dec0 20 remove_watcher() i=3 2019-12-09T12:34:07.109 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.106+0000 7f125037dec0 20 remove_watcher() i=4 2019-12-09T12:34:07.143 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.140+0000 7f125037dec0 20 remove_watcher() i=5 2019-12-09T12:34:07.188 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.185+0000 7f125037dec0 20 remove_watcher() i=6 2019-12-09T12:34:07.232 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.229+0000 7f125037dec0 20 remove_watcher() i=7 2019-12-09T12:34:07.243 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:34:07.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.360+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.392 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.390+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.393 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.390+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.399+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.399+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.407+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.407+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.430 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.428+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:07.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.642+0000 7fa22a551ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:34:07.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.642+0000 7fa22a551ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:34:07.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.642+0000 7fa22a551ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:34:07.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.642+0000 7fa22a551ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:34:07.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.642+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.650+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:07.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.650+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.658+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:07.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.658+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.673 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.670+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:07.673 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.670+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.678+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:07.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.678+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.686 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.684+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.687 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.684+0000 7fa22a551ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:34:07.687 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.684+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.691+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.691+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.702+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:07.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.702+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.709+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:07.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.709+0000 7fa22a551ec0 20 zonegroup default 2019-12-09T12:34:07.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.709+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.714+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.714+0000 7fa22a551ec0 10 Cannot find current period zone using local zone 2019-12-09T12:34:07.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.714+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.718+0000 7fa22a551ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:07.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.718+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.730+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:07.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.730+0000 7fa22a551ec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:07.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.737+0000 7fa22a551ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:07.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.737+0000 7fa22a551ec0 20 zone default found 2019-12-09T12:34:07.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.737+0000 7fa22a551ec0 20 started sync module instance, tier type = 2019-12-09T12:34:07.741 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:07.737+0000 7fa22a551ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:34:08.074 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=0 2019-12-09T12:34:08.074 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=1 2019-12-09T12:34:08.074 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=2 2019-12-09T12:34:08.074 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=3 2019-12-09T12:34:08.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=4 2019-12-09T12:34:08.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=5 2019-12-09T12:34:08.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.071+0000 7fa22a551ec0 20 add_watcher() i=6 2019-12-09T12:34:08.105 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.102+0000 7fa22a551ec0 20 add_watcher() i=7 2019-12-09T12:34:08.105 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.102+0000 7fa22a551ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:34:08.105 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.102+0000 7fa1e7fdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:34:08.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.111+0000 7fa216f7a700 0 -- 172.21.15.174:0/3726262480 >> v2:172.21.15.106:3300/0 conn(0x555dc2161440 msgr2=0x555dc2163860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T12:34:08.316 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.314+0000 7fa216f7a700 0 --2- 172.21.15.174:0/3726262480 >> v2:172.21.15.106:3300/0 conn(0x555dc2161440 0x555dc2163860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T12:34:08.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:08.356+0000 7fa216f7a700 0 -- 172.21.15.174:0/536938836 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x555dc21684f0 msgr2=0x555dc216a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:34:09.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.810+0000 7fa1567fc700 20 reqs_thread_entry: start 2019-12-09T12:34:09.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.810+0000 7fa14ffff700 20 reqs_thread_entry: start 2019-12-09T12:34:09.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.810+0000 7fa22a551ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:34:09.815 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.811+0000 7fa14e7fc700 20 reqs_thread_entry: start 2019-12-09T12:34:09.815 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.811+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:34:09.886 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.882+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:09.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.905+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:09.923 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:09.919+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.120 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.116+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:34:10.178 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.174+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.198 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.194+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.208 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.204+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.256+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:34:10.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.311+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.330+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.344 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.340+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.381+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:34:10.443 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.439+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.457+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.470 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.466+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.505+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:34:10.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.586+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.613+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.631 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.628+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.679+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:34:10.744 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.741+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.762+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.776 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.772+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.817+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:34:10.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.879+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.900+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:10.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.914+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:10.963 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:10.957+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:34:11.016 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.012+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.031 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.027+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.041 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.038+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.072+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:34:11.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.126+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.147 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.143+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.151+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.196+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:34:11.251 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.247+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.267 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.263+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.275 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.272+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.307+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:34:11.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.357+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.374+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.383+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.427+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:34:11.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.475+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.491+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.498+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.538+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:34:11.586 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.582+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.613 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.609+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.622+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.669 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.665+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:34:11.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.711+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.736+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.744+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.778+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:34:11.849 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.845+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.869+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:11.886 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.882+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:11.931 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.928+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:34:11.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:11.988+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.018 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.014+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.054+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.096+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:34:12.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.153+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.183 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.179+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.200+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.244+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:34:12.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.326+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.345+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.358 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.354+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.403+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:34:12.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.488+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.510+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.519+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.597+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:34:12.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.694+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.721+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.733+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.775 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.772+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:34:12.817 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.813+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.834 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.830+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.838+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.875 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.871+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:34:12.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.914+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.931+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:12.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.941+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:12.981 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:12.977+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:34:13.043 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.039+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.065 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.058+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.069+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.119 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.115+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:34:13.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.172+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.198+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.211 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.208+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.257 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.253+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:34:13.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.334+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.373 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.369+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.390 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.387+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.446+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:34:13.511 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.504+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.522+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.539 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.535+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.577 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.573+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:34:13.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.655+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.679+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.689+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.737+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:34:13.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.802+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.829+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:13.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.842+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:13.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.887+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:34:13.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:13.970+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.004 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.000+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:14.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.015+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.067+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:34:14.126 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.122+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.153 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.149+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:14.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.161+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.212+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:34:14.281 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.277+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.299 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.296+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:14.313 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.309+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.351+0000 7fa22a551ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:34:14.402 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.399+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.420 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.416+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:14.434 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.430+0000 7fa22a551ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:14.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.521+0000 7fa22a551ec0 20 remove_watcher() i=0 2019-12-09T12:34:14.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.521+0000 7fa22a551ec0 2 removed watcher, disabling cache 2019-12-09T12:34:14.571 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.567+0000 7fa22a551ec0 20 remove_watcher() i=1 2019-12-09T12:34:14.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.622+0000 7fa22a551ec0 20 remove_watcher() i=2 2019-12-09T12:34:14.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.687+0000 7fa22a551ec0 20 remove_watcher() i=3 2019-12-09T12:34:14.760 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.756+0000 7fa22a551ec0 20 remove_watcher() i=4 2019-12-09T12:34:14.810 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.806+0000 7fa22a551ec0 20 remove_watcher() i=5 2019-12-09T12:34:14.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.852+0000 7fa22a551ec0 20 remove_watcher() i=6 2019-12-09T12:34:14.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:14.897+0000 7fa22a551ec0 20 remove_watcher() i=7 2019-12-09T12:34:14.911 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:34:15.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.016+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.041+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.041+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.053 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.050+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.054 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.050+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.056+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.060 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.056+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.077 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.069+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:15.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.233+0000 7fe52883cec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:34:15.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.233+0000 7fe52883cec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:34:15.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.233+0000 7fe52883cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:34:15.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.233+0000 7fe52883cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:34:15.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.233+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.250 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.246+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:15.251 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.247+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.262+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:15.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.262+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.282 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.279+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:15.283 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.279+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.296 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.293+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:15.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.293+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.304+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.304+0000 7fe52883cec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:34:15.308 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.304+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.316+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.316+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.333+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:15.336 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.333+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.343+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:15.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.343+0000 7fe52883cec0 20 zonegroup default 2019-12-09T12:34:15.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.343+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.353+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.353+0000 7fe52883cec0 10 Cannot find current period zone using local zone 2019-12-09T12:34:15.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.353+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.363+0000 7fe52883cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:15.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.363+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.379+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:15.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.379+0000 7fe52883cec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:15.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.391+0000 7fe52883cec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:15.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.391+0000 7fe52883cec0 20 zone default found 2019-12-09T12:34:15.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.391+0000 7fe52883cec0 20 started sync module instance, tier type = 2019-12-09T12:34:15.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.391+0000 7fe52883cec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:34:15.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=0 2019-12-09T12:34:15.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=1 2019-12-09T12:34:15.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=2 2019-12-09T12:34:15.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=3 2019-12-09T12:34:15.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=4 2019-12-09T12:34:15.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=5 2019-12-09T12:34:15.739 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.734+0000 7fe52883cec0 20 add_watcher() i=6 2019-12-09T12:34:15.769 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.765+0000 7fe52883cec0 20 add_watcher() i=7 2019-12-09T12:34:15.769 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.765+0000 7fe52883cec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:34:15.769 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.765+0000 7fe4e67dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:34:15.776 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.771+0000 7fe50d265700 0 -- 172.21.15.174:0/393910127 >> v2:172.21.15.106:3300/0 conn(0x562d8f710440 msgr2=0x562d8f712860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:34:15.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:15.986+0000 7fe50d265700 0 --2- 172.21.15.174:0/393910127 >> v2:172.21.15.106:3300/0 conn(0x562d8f710440 0x562d8f712860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T12:34:16.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:16.095+0000 7fe516267700 0 -- 172.21.15.174:0/1395209060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe458002430 msgr2=0x7fe458002840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:34:17.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.320+0000 7fe45cff9700 20 reqs_thread_entry: start 2019-12-09T12:34:17.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.320+0000 7fe4567fc700 20 reqs_thread_entry: start 2019-12-09T12:34:17.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.320+0000 7fe52883cec0 20 init_complete bucket index max shards: 0 2019-12-09T12:34:17.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.321+0000 7fe454ff9700 20 reqs_thread_entry: start 2019-12-09T12:34:17.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.321+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:34:17.366 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.363+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.380+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.390+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.423+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:34:17.473 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.468+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.486+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.499+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.528+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:34:17.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.581+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.600 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.596+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.605+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.647 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.644+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:34:17.696 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.692+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.708+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.716+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.756 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.752+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:34:17.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.808+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.837 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.833+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.850 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.846+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.893 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.890+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:34:17.961 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.957+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:17.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.978+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:17.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:17.989+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.049 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.046+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:34:18.171 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.167+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.196 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.191+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:18.207 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.202+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.251+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:34:18.315 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.312+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.353 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.349+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:18.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.362+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.521+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:34:18.578 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.574+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.598+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:18.615 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.611+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.668+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:34:18.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.708+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.726 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.723+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:18.734 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.730+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.785 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.775+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:34:18.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.841+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.866 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.862+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:18.876 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.872+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:18.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:18.927+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:34:19.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.004+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.024+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.033+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.074+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:34:19.141 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.138+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.168+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.181 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.177+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.214+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:34:19.308 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.304+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.333 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.329+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.350+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.391+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:34:19.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.452+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.477+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.485+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.522+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:34:19.578 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.574+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.601+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.618 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.615+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.659+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:34:19.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.711+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.731 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.727+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.735+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.764+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:34:19.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.821+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.852 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.844+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.861 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.857+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.898 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.894+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:34:19.949 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.945+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:19.972 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.966+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:19.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:19.986+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.030 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.026+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:34:20.073 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.070+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.096+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.110+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.145+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:34:20.197 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.193+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.229 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.225+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.234+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.269+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:34:20.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.310+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.334+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.350 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.346+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.379+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:34:20.424 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.420+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.440 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.436+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.445+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.480+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:34:20.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.521+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.540 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.537+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.546+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.577+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:34:20.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.626+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.648+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.657+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.709 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.706+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:34:20.758 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.754+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.811 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.769+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.811 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.778+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.821+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:34:20.862 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.858+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.873+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:20.884 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.880+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.919 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.915+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:34:20.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.970+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:20.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:20.989+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:21.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.002+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.050 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.047+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:34:21.098 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.094+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.111+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:21.123 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.119+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.154+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:34:21.199 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.195+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.214 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.211+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:21.225 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.222+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.259 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.255+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:34:21.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.300+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.315+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:21.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.322+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.357+0000 7fe52883cec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:34:21.408 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.404+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.424+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:21.438 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.435+0000 7fe52883cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:21.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.528+0000 7fe52883cec0 20 remove_watcher() i=0 2019-12-09T12:34:21.537 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.528+0000 7fe52883cec0 2 removed watcher, disabling cache 2019-12-09T12:34:21.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.579+0000 7fe52883cec0 20 remove_watcher() i=1 2019-12-09T12:34:21.622 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.618+0000 7fe52883cec0 20 remove_watcher() i=2 2019-12-09T12:34:21.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.649+0000 7fe52883cec0 20 remove_watcher() i=3 2019-12-09T12:34:21.687 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.683+0000 7fe52883cec0 20 remove_watcher() i=4 2019-12-09T12:34:21.751 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.744+0000 7fe52883cec0 20 remove_watcher() i=5 2019-12-09T12:34:21.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.776+0000 7fe52883cec0 20 remove_watcher() i=6 2019-12-09T12:34:21.837 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.827+0000 7fe52883cec0 20 remove_watcher() i=7 2019-12-09T12:34:21.845 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:34:21.977 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:21.973+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.021 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.018+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.022 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.018+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.023+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.023+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.035 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.031+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.035 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.031+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.047+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:22.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.219+0000 7f09090ebec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:34:22.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.219+0000 7f09090ebec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:34:22.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.219+0000 7f09090ebec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:34:22.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.219+0000 7f09090ebec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:34:22.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.219+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.233+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:22.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.233+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.244+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:22.248 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.244+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.257 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.253+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:22.257 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.253+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.261+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:22.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.261+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.269+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.273 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.269+0000 7f09090ebec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:34:22.273 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.269+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.275+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.275+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.289+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:22.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.289+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.299+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:34:22.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.299+0000 7f09090ebec0 20 zonegroup default 2019-12-09T12:34:22.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.299+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.305+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.305+0000 7f09090ebec0 10 Cannot find current period zone using local zone 2019-12-09T12:34:22.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.305+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.315 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.311+0000 7f09090ebec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:34:22.315 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.311+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.323+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:34:22.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.323+0000 7f09090ebec0 20 rados->read ofs=0 len=0 2019-12-09T12:34:22.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.334+0000 7f09090ebec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:34:22.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.334+0000 7f09090ebec0 20 zone default found 2019-12-09T12:34:22.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.334+0000 7f09090ebec0 20 started sync module instance, tier type = 2019-12-09T12:34:22.338 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.334+0000 7f09090ebec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:34:22.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=0 2019-12-09T12:34:22.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=1 2019-12-09T12:34:22.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=2 2019-12-09T12:34:22.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=3 2019-12-09T12:34:22.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=4 2019-12-09T12:34:22.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=5 2019-12-09T12:34:22.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.687+0000 7f09090ebec0 20 add_watcher() i=6 2019-12-09T12:34:22.728 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.724+0000 7f09090ebec0 20 add_watcher() i=7 2019-12-09T12:34:22.729 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.724+0000 7f09090ebec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:34:22.729 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.725+0000 7f08c6fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:34:22.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.733+0000 7f08f6315700 0 -- 172.21.15.174:0/2583502065 >> v1:172.21.15.106:6789/0 conn(0x560086b309b0 legacy=0x560086b32dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:34:22.765 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:22.761+0000 7f08f6b16700 0 -- 172.21.15.174:0/2038053704 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x560086b1e2b0 msgr2=0x560086b31b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:34:24.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.314+0000 7f08357fa700 20 reqs_thread_entry: start 2019-12-09T12:34:24.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.314+0000 7f082effd700 20 reqs_thread_entry: start 2019-12-09T12:34:24.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.314+0000 7f09090ebec0 20 init_complete bucket index max shards: 0 2019-12-09T12:34:24.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.315+0000 7f082d7fa700 20 reqs_thread_entry: start 2019-12-09T12:34:24.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.315+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:34:24.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.357+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.376 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.373+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:24.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.382+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.414+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:34:24.468 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.464+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.483 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.479+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:24.491 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.488+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.526+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:34:24.597 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.593+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.638 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.635+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:24.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.648+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.695 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.691+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:34:24.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.764+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.785+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:24.798 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.794+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.832 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.828+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:34:24.884 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.880+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.903+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:24.916 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.912+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:24.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:24.955+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:34:25.014 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.009+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.034 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.030+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.043 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.040+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.087 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.083+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:34:25.139 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.135+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.163 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.159+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.174 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.170+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.214 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.210+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:34:25.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.270+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.299 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.295+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.308+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.365 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.361+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:34:25.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.412+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.484 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.481+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.489+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.531+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:34:25.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.586+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.605+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.617 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.613+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.676+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:34:25.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.723+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.740+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.749+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.834+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:34:25.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.898+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.918+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:25.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.927+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:25.987 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:25.983+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:34:26.135 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.132+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.159 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.155+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:26.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.169+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.213+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:34:26.273 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.269+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.299 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.295+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:26.313 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.309+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.353+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:34:26.438 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.434+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.470 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.467+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:26.486 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.482+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.532+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:34:26.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.621+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.652+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:26.671 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.667+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.733+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:34:26.827 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.823+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.853+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:26.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.868+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:26.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:26.955+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:34:27.044 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.040+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.063 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.059+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.071+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.114+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:34:27.181 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.178+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.197+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.209+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.256+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:34:27.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.317+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.340+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.354+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.394+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:34:27.473 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.469+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.491+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.508 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.505+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.553 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.549+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:34:27.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.653+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.679 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.676+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.686+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.723+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:34:27.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.785+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.815 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.810+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.821+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.856+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:34:27.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.917+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.936+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:27.949 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.945+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:27.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:27.983+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:34:28.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.023+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.049 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.039+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.048+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.131 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.124+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:34:28.169 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.164+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.182+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.197 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.193+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.225 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.222+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:34:28.284 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.280+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.307 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.302+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.316 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.313+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.368 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.363+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:34:28.421 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.417+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.433+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.448 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.444+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.493+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:34:28.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.553+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.582+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.598 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.594+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.639 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.635+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:34:28.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.695+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.714+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.728 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.725+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.777 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.774+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:34:28.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.820+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.837+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.850 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.846+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.884 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.880+0000 7f09090ebec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:34:28.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.938+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:28.966 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.961+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:34:28.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:28.973+0000 7f09090ebec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:34:29.056 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.052+0000 7f09090ebec0 20 remove_watcher() i=0 2019-12-09T12:34:29.056 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.052+0000 7f09090ebec0 2 removed watcher, disabling cache 2019-12-09T12:34:29.123 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.120+0000 7f09090ebec0 20 remove_watcher() i=1 2019-12-09T12:34:29.193 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.189+0000 7f09090ebec0 20 remove_watcher() i=2 2019-12-09T12:34:29.237 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.234+0000 7f09090ebec0 20 remove_watcher() i=3 2019-12-09T12:34:29.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.271+0000 7f09090ebec0 20 remove_watcher() i=4 2019-12-09T12:34:29.315 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.312+0000 7f09090ebec0 20 remove_watcher() i=5 2019-12-09T12:34:29.364 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.361+0000 7f09090ebec0 20 remove_watcher() i=6 2019-12-09T12:34:29.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:34:29.400+0000 7f09090ebec0 20 remove_watcher() i=7 2019-12-09T12:34:29.414 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-12-09T12:34:30.054 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :clean 2019-12-09T12:34:30.752 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileJava 2019-12-09T12:34:30.752 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processResources 2019-12-09T12:34:30.753 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :classes 2019-12-09T12:34:31.054 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:34:31.054 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileTestJava 2019-12-09T12:34:31.054 INFO:teuthology.orchestra.run.smithi174.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T12:34:31.054 INFO:teuthology.orchestra.run.smithi174.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T12:34:31.054 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:34:31.056 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T12:34:31.056 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :testClasses 2019-12-09T12:34:43.153 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:34:43.153 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :test 2019-12-09T12:34:43.154 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:34:43.154 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-12-09T12:34:55.262 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:13:47.114 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.276 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:13:47.128 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.293 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:13:47.118 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.294 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:13:47.119 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.297 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:13:47.122 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.302 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:13:47.127 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.483 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:13:47.307 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.483 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:13:47.334 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.493 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:13:47.317 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:34:55.651 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:34:55.652 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-12-09T12:36:16.351 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:36:16.353 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-12-09T12:36:21.452 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:36:21.452 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-12-09T12:37:10.620 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:16:02.472 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.622 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:16:02.446 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.626 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:16:02.478 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.650 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:16:02.475 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.653 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:16:02.478 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.657 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:16:02.481 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.718 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:16:02.538 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.718 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:16:02.540 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:10.723 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:16:02.574 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:37:24.951 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:37:24.951 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-12-09T12:37:29.851 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:37:29.851 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-12-09T12:38:18.251 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:38:18.253 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-12-09T12:38:41.051 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:38:41.051 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-12-09T12:39:04.051 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:39:04.052 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-12-09T12:39:16.951 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:39:16.951 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-12-09T12:39:53.151 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:39:53.250 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-12-09T12:39:58.151 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:39:58.151 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-12-09T12:40:06.438 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:18:58.290 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.449 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:18:58.300 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.456 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:18:58.274 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.456 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:18:58.280 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.457 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:18:58.275 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.457 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:18:58.279 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.554 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:18:58.379 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.572 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:18:58.397 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:06.608 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:18:58.460 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:40:10.852 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:10.852 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-12-09T12:40:23.851 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:23.855 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-12-09T12:40:40.854 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:40.857 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-12-09T12:40:43.152 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:43.152 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-12-09T12:40:50.051 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:50.052 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-12-09T12:40:50.352 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:50.352 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-12-09T12:40:52.107 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:52.107 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------- 2019-12-09T12:40:52.108 INFO:teuthology.orchestra.run.smithi174.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-12-09T12:40:52.108 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------- 2019-12-09T12:40:52.108 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:40:52.108 INFO:teuthology.orchestra.run.smithi174.stdout:BUILD SUCCESSFUL in 6m 22s 2019-12-09T12:40:52.108 INFO:teuthology.orchestra.run.smithi174.stdout:5 actionable tasks: 5 executed 2019-12-09T12:40:52.113 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:40:52.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.231+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.258+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.258+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.267+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.267+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.274+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.274+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.294+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:40:52.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.484+0000 7f7d15b17ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:40:52.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.484+0000 7f7d15b17ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:40:52.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.484+0000 7f7d15b17ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:40:52.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.484+0000 7f7d15b17ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:40:52.489 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.484+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.497+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:40:52.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.497+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.509+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:40:52.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.510+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.522+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:40:52.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.522+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.532+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:40:52.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.532+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.548 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.545+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.548 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.545+0000 7f7d15b17ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:40:52.549 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.545+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.555 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.551+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.555 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.551+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.562+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:40:52.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.562+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.571+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:40:52.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.571+0000 7f7d15b17ec0 20 zonegroup default 2019-12-09T12:40:52.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.571+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.579 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.576+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.576+0000 7f7d15b17ec0 10 Cannot find current period zone using local zone 2019-12-09T12:40:52.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.576+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.581+0000 7f7d15b17ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:52.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.581+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.595 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.591+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:40:52.595 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.591+0000 7f7d15b17ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:52.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.599+0000 7f7d15b17ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:40:52.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.599+0000 7f7d15b17ec0 20 zone default found 2019-12-09T12:40:52.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.599+0000 7f7d15b17ec0 20 started sync module instance, tier type = 2019-12-09T12:40:52.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.599+0000 7f7d15b17ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=0 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=1 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=2 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=3 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=4 2019-12-09T12:40:53.002 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=5 2019-12-09T12:40:53.003 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:52.998+0000 7f7d15b17ec0 20 add_watcher() i=6 2019-12-09T12:40:53.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:53.028+0000 7f7d15b17ec0 20 add_watcher() i=7 2019-12-09T12:40:53.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:53.028+0000 7f7d15b17ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:40:53.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:53.028+0000 7f7cd37de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:40:53.035 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:53.032+0000 7f7d03542700 0 -- 172.21.15.174:0/3174152161 >> v2:172.21.15.174:3300/0 conn(0x55e93efba4a0 msgr2=0x55e93efb9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:40:53.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:53.068+0000 7f7cfbfff700 0 -- 172.21.15.174:0/2761759954 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55e93efbae30 msgr2=0x55e93efbb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:40:54.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.506+0000 7f7c41ffb700 20 reqs_thread_entry: start 2019-12-09T12:40:54.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.507+0000 7f7c3b7fe700 20 reqs_thread_entry: start 2019-12-09T12:40:54.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.507+0000 7f7d15b17ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:40:54.511 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.507+0000 7f7c39ffb700 20 reqs_thread_entry: start 2019-12-09T12:40:54.511 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.507+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:40:54.564 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.560+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.580+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:54.596 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.592+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.647+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:40:54.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.702+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.741 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.737+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:54.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.748+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.797+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:40:54.919 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.915+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.938+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:54.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.949+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:54.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:54.982+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:40:55.048 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.044+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.068 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.065+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.081 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.077+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.114+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:40:55.211 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.208+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.246 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.242+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.260 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.256+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.339 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.336+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:40:55.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.402+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.424+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.439 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.435+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.477+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:40:55.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.540+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.564+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.578 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.575+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.623+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:40:55.683 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.679+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.706 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.702+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.715+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.763+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:40:55.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.814+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.836 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.833+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.843+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.885+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:40:55.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.948+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:55.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.971+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:55.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:55.983+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.033 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.029+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:40:56.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.107+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.135 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.132+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.150 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.146+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.233 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.228+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:40:56.291 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.285+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.315+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.330+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.376 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.372+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:40:56.435 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.432+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.459 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.455+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.468+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.503+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:40:56.565 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.558+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.579 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.576+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.587+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.641+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:40:56.722 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.719+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.748+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.763+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.815 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.811+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:40:56.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.886+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.910+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:56.924 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.920+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:56.954 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.950+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:40:57.009 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:56.998+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.016+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.034+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.188+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:40:57.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.237+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.280 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.277+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.306 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.304+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.353 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.349+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:40:57.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.401+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.420 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.417+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.426+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.465 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.462+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:40:57.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.510+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.529+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.538+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.580+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:40:57.640 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.636+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.662+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.674+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.739 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.732+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:40:57.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.795+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.815+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.827+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.871 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.866+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:40:57.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.931+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:57.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.954+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:57.968 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:57.965+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.033 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.028+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:40:58.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.099+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.127+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.148 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.145+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.197 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.194+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:40:58.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.258+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.283 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.281+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.290+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.330 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.328+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:40:58.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.384+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.406+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.419 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.416+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.459 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.457+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:40:58.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.497+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.516 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.513+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.522+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.559+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:40:58.619 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.615+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.639 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.637+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.650 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.647+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.687+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:40:58.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.746+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.770 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.767+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.782+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.835 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.833+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:40:58.898 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.896+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:58.924 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.921+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:58.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.934+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:59.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:58.998+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:40:59.070 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.067+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:59.094 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.092+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:59.107 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.105+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:59.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.143+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:40:59.188 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.186+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:59.209 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.206+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:40:59.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.216+0000 7f7d15b17ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:40:59.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.323+0000 7f7d15b17ec0 20 remove_watcher() i=0 2019-12-09T12:40:59.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.323+0000 7f7d15b17ec0 2 removed watcher, disabling cache 2019-12-09T12:40:59.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.405+0000 7f7d15b17ec0 20 remove_watcher() i=1 2019-12-09T12:40:59.468 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.466+0000 7f7d15b17ec0 20 remove_watcher() i=2 2019-12-09T12:40:59.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.512+0000 7f7d15b17ec0 20 remove_watcher() i=3 2019-12-09T12:40:59.565 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.562+0000 7f7d15b17ec0 20 remove_watcher() i=4 2019-12-09T12:40:59.637 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.630+0000 7f7d15b17ec0 20 remove_watcher() i=5 2019-12-09T12:40:59.669 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.666+0000 7f7d15b17ec0 20 remove_watcher() i=6 2019-12-09T12:40:59.747 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.744+0000 7f7d15b17ec0 20 remove_watcher() i=7 2019-12-09T12:40:59.757 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:40:59.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.863+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:59.894 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.892+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:59.894 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.892+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.897+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:59.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.897+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:59.905 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.903+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:40:59.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.903+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:40:59.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:40:59.920+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:00.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.075+0000 7fb3a0da6ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:41:00.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.075+0000 7fb3a0da6ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:41:00.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.075+0000 7fb3a0da6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:41:00.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.075+0000 7fb3a0da6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:41:00.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.075+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.094 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.091+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:00.094 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.091+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.099+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:00.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.100+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.108+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:00.110 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.108+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.122 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.119+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:00.122 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.119+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.125+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:00.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.125+0000 7fb3a0da6ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:41:00.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.125+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.135 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.132+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:00.135 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.132+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.144 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.141+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:00.144 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.141+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.149+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:00.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.149+0000 7fb3a0da6ec0 20 zonegroup default 2019-12-09T12:41:00.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.149+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.154+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:00.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.154+0000 7fb3a0da6ec0 10 Cannot find current period zone using local zone 2019-12-09T12:41:00.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.154+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.161 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.159+0000 7fb3a0da6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:00.161 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.159+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.166+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:00.168 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.166+0000 7fb3a0da6ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:00.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.173+0000 7fb3a0da6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:00.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.173+0000 7fb3a0da6ec0 20 zone default found 2019-12-09T12:41:00.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.173+0000 7fb3a0da6ec0 20 started sync module instance, tier type = 2019-12-09T12:41:00.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.173+0000 7fb3a0da6ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:41:00.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=0 2019-12-09T12:41:00.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=1 2019-12-09T12:41:00.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=2 2019-12-09T12:41:00.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=3 2019-12-09T12:41:00.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=4 2019-12-09T12:41:00.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=5 2019-12-09T12:41:00.498 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.494+0000 7fb3a0da6ec0 20 add_watcher() i=6 2019-12-09T12:41:00.528 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.525+0000 7fb3a0da6ec0 20 add_watcher() i=7 2019-12-09T12:41:00.528 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.525+0000 7fb3a0da6ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:41:00.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.525+0000 7fb35e7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:41:00.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.532+0000 7fb38e7d1700 0 -- 172.21.15.174:0/2144613081 >> v2:172.21.15.174:3300/0 conn(0x5641813b21f0 msgr2=0x5641813c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:41:00.737 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.735+0000 7fb38e7d1700 0 --2- 172.21.15.174:0/2144613081 >> v2:172.21.15.174:3300/0 conn(0x5641813b21f0 0x5641813c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T12:41:00.807 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:00.804+0000 7fb38d7cf700 0 -- 172.21.15.174:0/1591348410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5641813eecb0 msgr2=0x5641813d7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:41:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.234+0000 7fb2d4ff9700 20 reqs_thread_entry: start 2019-12-09T12:41:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.235+0000 7fb2ce7fc700 20 reqs_thread_entry: start 2019-12-09T12:41:02.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.235+0000 7fb3a0da6ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:41:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.236+0000 7fb2ccff9700 20 reqs_thread_entry: start 2019-12-09T12:41:02.239 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.236+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:41:02.292 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.289+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.306+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:02.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.316+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.354+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:41:02.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.405+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.423+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:02.435 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.433+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.469+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:41:02.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.528+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.543+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:02.554 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.552+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.601+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:41:02.686 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.684+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.720 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.717+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:02.730 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.727+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.781+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:41:02.861 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.858+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.885+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:02.902 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.899+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:02.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:02.968+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:41:03.040 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.038+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.072+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.087 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.084+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.124+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:41:03.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.200+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.227 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.224+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.238+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.288 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.286+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:41:03.364 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.362+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.409 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.401+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.420 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.418+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.476+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:41:03.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.553+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.586+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.601+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.664 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.661+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:41:03.729 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.726+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.746+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.756 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.754+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.791+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:41:03.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.843+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.861+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.873 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.870+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.904+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:41:03.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.952+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:03.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.970+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:03.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:03.979+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.018 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.015+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:41:04.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.076+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.100+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.120+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.183 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.179+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:41:04.250 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.248+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.283 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.274+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.286+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.390 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.388+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:41:04.443 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.441+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.462 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.460+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.474 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.471+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.522+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:41:04.594 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.592+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.621 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.619+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.635+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.711+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:41:04.781 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.762+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.780+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.795 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.792+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.830+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:41:04.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.919+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:04.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.939+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:04.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:04.955+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.011 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.008+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:41:05.073 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.070+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.098 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.096+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.105+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.169+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:41:05.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.240+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.267 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.265+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.281 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.278+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.320+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:41:05.386 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.384+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.415 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.413+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.426+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.479 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.477+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:41:05.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.529+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.553 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.550+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.563 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.560+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.599+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:41:05.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.660+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.683+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.703+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.743+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:41:05.804 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.802+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.826+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.839+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.890+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:41:05.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.955+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:05.981 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.979+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:05.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:05.992+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.034+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:41:06.090 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.087+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.110+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.121+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.163 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.160+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:41:06.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.214+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.237+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.259 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.257+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.339+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:41:06.405 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.403+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.420+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.436 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.433+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.482+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:41:06.565 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.558+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.580 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.575+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.584+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.625+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:41:06.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.672+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.691 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.688+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.699 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.696+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.741 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.738+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:41:06.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.840+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.865+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:06.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.880+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:06.938 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:06.935+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:41:07.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.033+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:07.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.061+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:07.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.074+0000 7fb3a0da6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.198+0000 7fb3a0da6ec0 20 remove_watcher() i=0 2019-12-09T12:41:07.201 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.198+0000 7fb3a0da6ec0 2 removed watcher, disabling cache 2019-12-09T12:41:07.248 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.246+0000 7fb3a0da6ec0 20 remove_watcher() i=1 2019-12-09T12:41:07.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.291+0000 7fb3a0da6ec0 20 remove_watcher() i=2 2019-12-09T12:41:07.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.344+0000 7fb3a0da6ec0 20 remove_watcher() i=3 2019-12-09T12:41:07.395 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.392+0000 7fb3a0da6ec0 20 remove_watcher() i=4 2019-12-09T12:41:07.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.426+0000 7fb3a0da6ec0 20 remove_watcher() i=5 2019-12-09T12:41:07.480 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.478+0000 7fb3a0da6ec0 20 remove_watcher() i=6 2019-12-09T12:41:07.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.530+0000 7fb3a0da6ec0 20 remove_watcher() i=7 2019-12-09T12:41:07.543 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:41:07.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.658+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.690+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.690+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.701+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.701+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.710+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.710+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.729+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.910+0000 7fbea8ba5ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:41:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.910+0000 7fbea8ba5ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:41:07.913 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.910+0000 7fbea8ba5ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:41:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.910+0000 7fbea8ba5ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:41:07.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.910+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.923+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:07.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.923+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.931+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:07.935 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.932+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.942+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:07.946 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.942+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.949+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:07.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.949+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.957+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.957+0000 7fbea8ba5ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:41:07.960 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.957+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.965 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.961+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.965 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.961+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.972+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:07.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.972+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.979+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.979+0000 7fbea8ba5ec0 20 zonegroup default 2019-12-09T12:41:07.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.979+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.987+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.987+0000 7fbea8ba5ec0 10 Cannot find current period zone using local zone 2019-12-09T12:41:07.989 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.987+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:07.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.992+0000 7fbea8ba5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:07.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:07.992+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.002+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:08.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.002+0000 7fbea8ba5ec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:08.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.010+0000 7fbea8ba5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:08.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.010+0000 7fbea8ba5ec0 20 zone default found 2019-12-09T12:41:08.013 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.010+0000 7fbea8ba5ec0 20 started sync module instance, tier type = 2019-12-09T12:41:08.013 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.010+0000 7fbea8ba5ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=0 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=1 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=2 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=3 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=4 2019-12-09T12:41:08.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.414+0000 7fbea8ba5ec0 20 add_watcher() i=5 2019-12-09T12:41:08.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.415+0000 7fbea8ba5ec0 20 add_watcher() i=6 2019-12-09T12:41:08.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.447+0000 7fbea8ba5ec0 20 add_watcher() i=7 2019-12-09T12:41:08.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.447+0000 7fbea8ba5ec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:41:08.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.447+0000 7fbe667dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:41:08.456 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.453+0000 7fbe95dcf700 0 -- 172.21.15.174:0/1619910931 >> v2:172.21.15.174:3300/0 conn(0x5619014e5880 msgr2=0x5619015021c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:41:08.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:08.496+0000 7fbe955ce700 0 -- 172.21.15.174:0/1457487000 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x561901516d70 msgr2=0x561901519190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T12:41:10.248 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.246+0000 7fbddcff9700 20 reqs_thread_entry: start 2019-12-09T12:41:10.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.246+0000 7fbdd67fc700 20 reqs_thread_entry: start 2019-12-09T12:41:10.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.246+0000 7fbea8ba5ec0 20 init_complete bucket index max shards: 0 2019-12-09T12:41:10.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.246+0000 7fbdd4ff9700 20 reqs_thread_entry: start 2019-12-09T12:41:10.249 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.246+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:41:10.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.311+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.338+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:10.357 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.355+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.411 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.409+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:41:10.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.494+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.520+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:10.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.536+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.581+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:41:10.634 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.631+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.657 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.654+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:10.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.667+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.723+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:41:10.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.794+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.813+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:10.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.824+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.878 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.876+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:41:10.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.925+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.941+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:10.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.951+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:10.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:10.996+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:41:11.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.055+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.072+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.081 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.079+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.115+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:41:11.158 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.155+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.176 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.173+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.185 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.182+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.223 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.221+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:41:11.282 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.279+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.299 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.297+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.309+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.349 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.346+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:41:11.437 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.434+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.463 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.461+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.476 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.473+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.516 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.514+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:41:11.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.565+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.581+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.593 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.590+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.628+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:41:11.680 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.678+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.695 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.693+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.700+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.753 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.751+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:41:11.811 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.809+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.826+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.836+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.875 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.873+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:41:11.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.920+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.939+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:11.951 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.948+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:11.990 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:11.987+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:41:12.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.029+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.050 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.047+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.061 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.059+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.107 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.104+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:41:12.180 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.177+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.198 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.195+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.206 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.203+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.263+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:41:12.331 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.329+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.349 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.347+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.358 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.355+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.392 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.384+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:41:12.440 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.437+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.457+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.472 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.469+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.505+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:41:12.556 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.553+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.571+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.581+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.613+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:41:12.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.658+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.696 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.694+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.706 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.704+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.742+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:41:12.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.780+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.801 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.798+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.810 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.807+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.843+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:41:12.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.906+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.930+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:12.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.942+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:12.980 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:12.972+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:41:13.048 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.046+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.108 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.106+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.125 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.122+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.166 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.163+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:41:13.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.210+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.233 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.231+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.248 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.246+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.284 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.282+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:41:13.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.339+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.372 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.369+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.382+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.422+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:41:13.466 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.463+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.479+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.491 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.488+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.540 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.538+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:41:13.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.600+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.620 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.617+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.631 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.629+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.679+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:41:13.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.757+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.787 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.784+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.798+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.853+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:41:13.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.934+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:13.962 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.959+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:13.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:13.971+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.036 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.033+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:41:14.120 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.117+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.151 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.148+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:14.169 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.167+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.229 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.227+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:41:14.289 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.286+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.313 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.310+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:14.332 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.330+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.367+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:41:14.417 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.414+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.437 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.429+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:14.443 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.437+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.485+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:41:14.548 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.545+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.577 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.575+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:14.591 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.589+0000 7fbea8ba5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:14.673 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.669+0000 7fbea8ba5ec0 20 remove_watcher() i=0 2019-12-09T12:41:14.673 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.669+0000 7fbea8ba5ec0 2 removed watcher, disabling cache 2019-12-09T12:41:14.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.708+0000 7fbea8ba5ec0 20 remove_watcher() i=1 2019-12-09T12:41:14.752 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.750+0000 7fbea8ba5ec0 20 remove_watcher() i=2 2019-12-09T12:41:14.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.822+0000 7fbea8ba5ec0 20 remove_watcher() i=3 2019-12-09T12:41:14.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.880+0000 7fbea8ba5ec0 20 remove_watcher() i=4 2019-12-09T12:41:14.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.941+0000 7fbea8ba5ec0 20 remove_watcher() i=5 2019-12-09T12:41:15.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:14.998+0000 7fbea8ba5ec0 20 remove_watcher() i=6 2019-12-09T12:41:15.044 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.039+0000 7fbea8ba5ec0 20 remove_watcher() i=7 2019-12-09T12:41:15.051 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T12:41:15.177 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.169+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.206 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.203+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.206 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.203+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.211+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.211+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.218+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.221 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.218+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.239 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.237+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:15.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.512+0000 7f10bf85eec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T12:41:15.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.512+0000 7f10bf85eec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T12:41:15.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.512+0000 7f10bf85eec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T12:41:15.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.512+0000 7f10bf85eec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T12:41:15.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.512+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.527+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:15.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.527+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.541+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:15.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.541+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.554+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:15.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.555+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.571 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.569+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:15.572 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.569+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.581 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.579+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.579+0000 7f10bf85eec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T12:41:15.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.579+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.590+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.590+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.608+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:15.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.608+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.624+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T12:41:15.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.624+0000 7f10bf85eec0 20 zonegroup default 2019-12-09T12:41:15.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.624+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.633+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.633+0000 7f10bf85eec0 10 Cannot find current period zone using local zone 2019-12-09T12:41:15.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.633+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.642+0000 7f10bf85eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T12:41:15.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.642+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.659+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T12:41:15.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.659+0000 7f10bf85eec0 20 rados->read ofs=0 len=0 2019-12-09T12:41:15.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.676+0000 7f10bf85eec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T12:41:15.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.676+0000 7f10bf85eec0 20 zone default found 2019-12-09T12:41:15.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.676+0000 7f10bf85eec0 20 started sync module instance, tier type = 2019-12-09T12:41:15.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:15.676+0000 7f10bf85eec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T12:41:16.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=0 2019-12-09T12:41:16.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=1 2019-12-09T12:41:16.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=2 2019-12-09T12:41:16.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=3 2019-12-09T12:41:16.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=4 2019-12-09T12:41:16.013 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.009+0000 7f10bf85eec0 20 add_watcher() i=5 2019-12-09T12:41:16.016 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.014+0000 7f10bf85eec0 20 add_watcher() i=6 2019-12-09T12:41:16.043 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.041+0000 7f10bf85eec0 20 add_watcher() i=7 2019-12-09T12:41:16.044 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.041+0000 7f10bf85eec0 2 all 8 watchers are set, enabling cache 2019-12-09T12:41:16.044 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.041+0000 7f107cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T12:41:16.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.043+0000 7f10ad289700 0 -- >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ca75e03800 msgr2=0x55ca75e02b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T12:41:16.089 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:16.086+0000 7f10a7fff700 0 -- 172.21.15.174:0/1710977247 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x55ca75e2fad0 msgr2=0x55ca75e31ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T12:41:17.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.581+0000 7f0feeffd700 20 reqs_thread_entry: start 2019-12-09T12:41:17.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.582+0000 7f0fecff9700 20 reqs_thread_entry: start 2019-12-09T12:41:17.584 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.582+0000 7f10bf85eec0 20 init_complete bucket index max shards: 0 2019-12-09T12:41:17.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.583+0000 7f0fe2ffd700 20 reqs_thread_entry: start 2019-12-09T12:41:17.585 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.583+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T12:41:17.667 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.664+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:17.696 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.693+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:17.707 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.705+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:17.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.759+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T12:41:17.829 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.826+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:17.857 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.854+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:17.870 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.867+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:17.916 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.913+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T12:41:17.966 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.964+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:17.981 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.979+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:17.989 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:17.986+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.034+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T12:41:18.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.109+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.137+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.155+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.197+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T12:41:18.271 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.268+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.296+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.312+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.350+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T12:41:18.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.412+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.446 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.443+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.458+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.521+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T12:41:18.595 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.592+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.625+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.641+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.688 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.685+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T12:41:18.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.730+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.746+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.764 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.761+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.793+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T12:41:18.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.839+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.862 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.859+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.872 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.870+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.906+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T12:41:18.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.948+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:18.966 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.963+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:18.973 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:18.970+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.019 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.016+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T12:41:19.073 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.071+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.089+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.099 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.097+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.138+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T12:41:19.194 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.192+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.211 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.208+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.217+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.262 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.260+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T12:41:19.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.318+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.338+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.352 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.349+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.404+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T12:41:19.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.474+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.507 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.505+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.522+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.573+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T12:41:19.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.631+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.653+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.666 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.664+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.701 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.698+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T12:41:19.763 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.761+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.781 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.779+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.793 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.788+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.834 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.831+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T12:41:19.884 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.882+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.899+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:19.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.907+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:19.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.952+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T12:41:20.001 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:19.998+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.015 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.013+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.026 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.023+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.062+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T12:41:20.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.114+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.136 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.133+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.148 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.146+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.187+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T12:41:20.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.238+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.254+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.265 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.262+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.295+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T12:41:20.349 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.346+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.364 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.362+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.373 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.371+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.408+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T12:41:20.488 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.485+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.510+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.528+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.579+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T12:41:20.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.651+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.681 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.679+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.695+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.744+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T12:41:20.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.814+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.835+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:20.848 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.846+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.896 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.893+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T12:41:20.972 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.969+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:20.992 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:20.989+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.003 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.001+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.043+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T12:41:21.123 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.121+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.147 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.144+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.159 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.157+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.203 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.200+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T12:41:21.295 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.292+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.316+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.333+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.381+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T12:41:21.490 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.487+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.517+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.544+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.597 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.595+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T12:41:21.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.654+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.671+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.681+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.715+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T12:41:21.781 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.779+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.803 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.801+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.817 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.813+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.869 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.866+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T12:41:21.938 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.936+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:21.974 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.971+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:21.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:21.986+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:22.029 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.027+0000 7f10bf85eec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T12:41:22.092 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.090+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:22.121 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.117+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T12:41:22.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.137+0000 7f10bf85eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T12:41:22.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.270+0000 7f10bf85eec0 20 remove_watcher() i=0 2019-12-09T12:41:22.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.270+0000 7f10bf85eec0 2 removed watcher, disabling cache 2019-12-09T12:41:22.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.324+0000 7f10bf85eec0 20 remove_watcher() i=1 2019-12-09T12:41:22.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.382+0000 7f10bf85eec0 20 remove_watcher() i=2 2019-12-09T12:41:22.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.420+0000 7f10bf85eec0 20 remove_watcher() i=3 2019-12-09T12:41:22.466 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.464+0000 7f10bf85eec0 20 remove_watcher() i=4 2019-12-09T12:41:22.508 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.506+0000 7f10bf85eec0 20 remove_watcher() i=5 2019-12-09T12:41:22.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.542+0000 7f10bf85eec0 20 remove_watcher() i=6 2019-12-09T12:41:22.583 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T12:41:22.581+0000 7f10bf85eec0 20 remove_watcher() i=7 2019-12-09T12:41:22.591 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-12-09T12:41:23.113 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :clean 2019-12-09T12:41:23.609 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileJava 2019-12-09T12:41:23.609 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processResources 2019-12-09T12:41:23.610 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :classes 2019-12-09T12:41:23.710 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :compileTestJava 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :processTestResources NO-SOURCE 2019-12-09T12:41:23.711 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :testClasses 2019-12-09T12:41:26.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:26.109 INFO:teuthology.orchestra.run.smithi174.stdout:> Task :test 2019-12-09T12:41:26.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:26.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-12-09T12:41:31.310 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:31.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-12-09T12:41:44.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:44.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-12-09T12:41:47.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:47.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-12-09T12:41:56.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:56.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-12-09T12:41:59.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:41:59.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-12-09T12:42:04.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:04.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-12-09T12:42:38.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:38.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-12-09T12:42:45.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:45.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-12-09T12:42:47.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:47.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-12-09T12:42:51.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:51.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-12-09T12:42:56.114 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:42:56.114 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-12-09T12:43:05.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:05.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-12-09T12:43:05.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:05.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-12-09T12:43:05.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:05.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-12-09T12:43:07.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:07.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-12-09T12:43:11.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:11.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-12-09T12:43:20.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:20.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-12-09T12:43:26.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:26.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-12-09T12:43:29.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:29.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-12-09T12:43:36.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:36.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-12-09T12:43:39.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:39.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-12-09T12:43:45.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:45.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-12-09T12:43:47.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:47.809 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-12-09T12:43:53.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:53.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-12-09T12:43:56.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:43:56.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-12-09T12:44:02.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:02.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-12-09T12:44:04.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:04.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-12-09T12:44:09.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:09.410 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-12-09T12:44:10.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:10.809 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-12-09T12:44:18.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:18.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-12-09T12:44:24.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:24.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-12-09T12:44:29.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:29.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-12-09T12:44:50.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:44:50.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-12-09T12:45:05.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:45:05.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-12-09T12:45:18.503 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:10.354 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.512 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:10.364 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.526 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:10.351 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.530 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:10.355 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.532 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:10.357 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.535 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:10.360 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.704 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:10.555 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.723 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:10.548 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:18.740 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:10.564 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.524 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:24.349 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.527 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:24.352 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.533 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:24.358 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.535 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:24.386 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.538 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:24.390 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.545 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:24.369 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.557 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:24:24.382 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.572 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:24:24.424 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:45:32.599 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:24:24.423 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:49:18.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:18.728 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-12-09T12:49:28.508 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:28.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-12-09T12:49:29.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:29.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-12-09T12:49:35.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:35.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-12-09T12:49:40.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:40.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-12-09T12:49:53.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:49:53.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-12-09T12:50:07.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:07.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-12-09T12:50:16.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:16.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-12-09T12:50:17.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:17.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-12-09T12:50:17.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:17.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-12-09T12:50:29.656 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:29:21.507 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.664 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:29:21.515 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.668 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:29:21.493 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:29:21.496 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.672 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:29:21.497 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.675 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:29:21.500 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.827 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:29:21.678 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.837 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:29:21.662 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:29.845 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:29:21.670 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:50:37.965 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T12:50:37.897+0000 fb3ed700 -1 handle_data failed with exit code -104 2019-12-09T12:50:40.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:40.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-12-09T12:50:47.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:50:47.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-12-09T12:51:03.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:51:03.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-12-09T12:51:18.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:51:18.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-12-09T12:51:50.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:51:50.814 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-12-09T12:51:58.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:51:58.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-12-09T12:53:54.381 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:32:46.231 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.390 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:32:46.241 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.432 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:32:46.257 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.450 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:32:46.275 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.453 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:32:46.278 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.469 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:32:46.291 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.522 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:32:46.373 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.548 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:32:46.373 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:53:54.559 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:32:46.383 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:10.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:55:10.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-12-09T12:55:22.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:55:22.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-12-09T12:55:39.849 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:34:31.699 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.855 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:34:31.706 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.858 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:34:31.683 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.864 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:34:31.689 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.866 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:34:31.690 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.871 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:34:31.695 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.945 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:34:31.797 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.959 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:34:31.784 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T12:55:39.997 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:34:31.822 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T12:56:02.910 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:56:02.910 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-12-09T12:56:10.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:56:10.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-12-09T12:56:44.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:56:44.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-12-09T12:56:52.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:56:52.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-12-09T12:57:02.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:02.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-12-09T12:57:13.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:13.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-12-09T12:57:13.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:13.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-12-09T12:57:15.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:15.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-12-09T12:57:16.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:16.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-12-09T12:57:34.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:34.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-12-09T12:57:39.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:39.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-12-09T12:57:54.817 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:54.817 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-12-09T12:57:59.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:57:59.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-12-09T12:58:29.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:58:29.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-12-09T12:58:36.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:58:36.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-12-09T12:59:04.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:59:04.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-12-09T12:59:18.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:59:18.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-12-09T12:59:48.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:59:48.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-12-09T12:59:53.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T12:59:53.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-12-09T13:00:49.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:00:49.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-12-09T13:00:50.863 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:39:42.714 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:50.867 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:39:42.719 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:50.871 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:39:42.696 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:50.875 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:39:42.700 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:50.880 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:39:42.705 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:50.884 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:39:42.709 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:51.044 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:39:42.869 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:51.081 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:39:42.932 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:00:51.088 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:39:42.913 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:01:07.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:01:07.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-12-09T13:01:52.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:01:52.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-12-09T13:02:16.266 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:08.117 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.313 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:08.165 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.360 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:08.185 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.363 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:08.187 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.364 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:08.188 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.371 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:08.216 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.372 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:08.194 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.413 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:08.237 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:02:16.421 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:08.246 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.779 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:53.630 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.786 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:53.638 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.850 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:53.675 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.856 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:53.680 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.870 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:41:53.721 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.883 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:53.707 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.888 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:53.713 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.900 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:41:53.725 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:01.935 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:41:53.760 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:03:08.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:03:08.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-12-09T13:05:07.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:05:07.822 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-12-09T13:06:00.984 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:44:52.835 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:00.992 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:44:52.842 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.005 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:44:52.821 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.005 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:44:52.826 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.005 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:44:52.824 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.006 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:44:52.827 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.107 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:44:52.932 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.112 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:44:52.936 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:06:01.143 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:44:52.994 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:07:15.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:07:15.910 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-12-09T13:10:38.489 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:49:30.340 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.513 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:49:30.314 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.514 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:49:30.316 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.514 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:49:30.354 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.522 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:49:30.346 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.531 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:49:30.356 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.628 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:49:30.453 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.635 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:49:30.487 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:10:38.639 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:49:30.464 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.234 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:50:05.085 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.245 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:50:05.090 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.250 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:50:05.075 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.253 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:50:05.078 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.256 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:50:05.081 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.257 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:50:05.082 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.300 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:50:05.124 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.305 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:50:05.129 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:13.306 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:50:05.157 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:11:23.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:11:23.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-12-09T13:13:43.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:13:43.110 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-12-09T13:16:25.802 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:55:17.653 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.817 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:55:17.640 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.817 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:55:17.664 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.818 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:55:17.643 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.819 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:55:17.643 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.824 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:55:17.648 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:25.988 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:55:17.839 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:26.025 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:55:17.850 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:16:26.034 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:55:17.858 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:17:49.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:17:49.234 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-12-09T13:19:00.477 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:57:52.329 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.518 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:57:52.370 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.559 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:57:52.384 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.560 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:57:52.384 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.561 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:57:52.386 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.565 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:57:52.390 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.593 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:01:57:52.444 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.645 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:01:57:52.469 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:19:00.656 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:01:57:52.480 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:20:16.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:20:16.410 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-12-09T13:21:36.123 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:00:27.974 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.133 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:00:27.958 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.133 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:00:27.984 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.136 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:00:27.961 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.137 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:00:27.962 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.142 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:00:27.967 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.235 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:00:28.086 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.267 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:00:28.065 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:21:36.267 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:00:28.074 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:31.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:26:31.923 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-12-09T13:26:49.412 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:05:41.262 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.425 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:05:41.249 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.425 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:05:41.276 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.430 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:05:41.255 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.440 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:05:41.265 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.445 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:05:41.270 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.604 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:05:41.428 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.640 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:05:41.491 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:26:49.651 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:05:41.476 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.428 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:06:14.278 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.440 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:06:14.265 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.441 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:06:14.266 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.460 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:06:14.311 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.471 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:06:14.296 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.489 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:06:14.313 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.492 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:06:14.317 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.500 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:06:14.351 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:27:22.522 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:06:14.346 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:28:41.810 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:28:41.810 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-12-09T13:32:00.576 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:10:52.427 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.584 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:10:52.435 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.595 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:10:52.419 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.603 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:10:52.427 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.604 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:10:52.428 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.606 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:10:52.430 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.761 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:10:52.585 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.771 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:10:52.595 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:32:00.783 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:10:52.634 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:33:14.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:33:14.516 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-12-09T13:35:14.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:35:14.210 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-12-09T13:35:44.434 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:14:36.285 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.475 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:14:36.300 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.477 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:14:36.302 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.483 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:14:36.334 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.497 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:14:36.321 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.512 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:14:36.337 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.578 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:14:36.403 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.619 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:14:36.443 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:35:44.687 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:14:36.538 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.138 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:16:03.988 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.149 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:16:04.000 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.154 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:16:03.978 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.157 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:16:03.981 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.160 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:16:03.983 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.160 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:16:03.984 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.230 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:16:04.054 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.236 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:16:04.087 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:37:12.245 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:16:04.069 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:38:30.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:38:30.410 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-12-09T13:39:45.713 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:39:45.715 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-12-09T13:41:24.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:41:24.712 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-12-09T13:41:41.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:41:41.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-12-09T13:41:53.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:41:53.909 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-12-09T13:42:02.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:02.010 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-12-09T13:42:16.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:16.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-12-09T13:42:24.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:24.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-12-09T13:42:24.353 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:21:16.204 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.361 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:21:16.212 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.371 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:21:16.196 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.372 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:21:16.196 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.377 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:21:16.202 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.378 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:21:16.203 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.545 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:21:16.396 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.581 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:21:16.406 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:24.620 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:21:16.444 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:42:34.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:34.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-12-09T13:42:44.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:44.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-12-09T13:42:57.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:42:57.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-12-09T13:43:05.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:05.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-12-09T13:43:13.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:13.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-12-09T13:43:19.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:19.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-12-09T13:43:26.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:26.309 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-12-09T13:43:33.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:33.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-12-09T13:43:43.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:43.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-12-09T13:43:52.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:43:52.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-12-09T13:44:03.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:03.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-12-09T13:44:06.366 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:22:58.217 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.379 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:22:58.230 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.399 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:22:58.224 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.409 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:22:58.233 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.411 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:22:58.235 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.417 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:22:58.241 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.449 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:22:58.300 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.477 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:22:58.301 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:06.482 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:22:58.306 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:10.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:10.010 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-12-09T13:44:11.682 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:23:03.532 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.689 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:23:03.540 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.721 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:23:03.545 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.725 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:23:03.575 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.743 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:23:03.568 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.747 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:23:03.571 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.759 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:23:03.583 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.764 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:23:03.589 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:11.776 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:23:03.601 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:44:20.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:20.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-12-09T13:44:27.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:27.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-12-09T13:44:39.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:39.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-12-09T13:44:45.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:45.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-12-09T13:44:58.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:44:58.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-12-09T13:45:05.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:05.810 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-12-09T13:45:18.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:18.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-12-09T13:45:26.510 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:26.510 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-12-09T13:45:37.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:37.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-12-09T13:45:44.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:44.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-12-09T13:45:55.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:45:55.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-12-09T13:46:02.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:02.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-12-09T13:46:14.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:14.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-12-09T13:46:17.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:17.610 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-12-09T13:46:22.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:22.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-12-09T13:46:26.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:26.210 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-12-09T13:46:33.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:33.510 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-12-09T13:46:36.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:36.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-12-09T13:46:45.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:45.410 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-12-09T13:46:48.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:48.809 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-12-09T13:46:56.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:46:56.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-12-09T13:47:06.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:06.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-12-09T13:47:11.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:11.510 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-12-09T13:47:12.209 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:12.209 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-12-09T13:47:13.109 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:13.109 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-12-09T13:47:17.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:17.610 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-12-09T13:47:26.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:26.410 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-12-09T13:47:30.709 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:30.709 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-12-09T13:47:37.430 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:26:29.281 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.439 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:26:29.290 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.446 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:26:29.270 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.448 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:26:29.273 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.453 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:26:29.277 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.454 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:26:29.279 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.580 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:26:29.404 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.583 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:26:29.407 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:37.606 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:26:29.457 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:47:38.509 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:38.509 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-12-09T13:47:40.409 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:40.409 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-12-09T13:47:47.009 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:47.009 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-12-09T13:47:49.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:49.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-12-09T13:47:55.609 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:47:55.609 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-12-09T13:48:00.309 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:00.310 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-12-09T13:48:08.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:08.810 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-12-09T13:48:22.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:22.910 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-12-09T13:48:30.710 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:30.710 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-12-09T13:48:45.809 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:45.809 INFO:teuthology.orchestra.run.smithi174.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-12-09T13:48:57.748 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:57.748 INFO:teuthology.orchestra.run.smithi174.stdout:---------------------------------------------------------------------- 2019-12-09T13:48:57.748 INFO:teuthology.orchestra.run.smithi174.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-12-09T13:48:57.748 INFO:teuthology.orchestra.run.smithi174.stdout:---------------------------------------------------------------------- 2019-12-09T13:48:57.749 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:48:57.749 INFO:teuthology.orchestra.run.smithi174.stdout:BUILD SUCCESSFUL in 1h 7m 35s 2019-12-09T13:48:57.749 INFO:teuthology.orchestra.run.smithi174.stdout:5 actionable tasks: 5 executed 2019-12-09T13:48:57.762 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T13:48:57.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.904+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:57.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.942+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:57.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.942+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:57.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.949+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:57.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.949+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:57.963 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.960+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:57.963 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.960+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:57.983 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:57.980+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:58.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.172+0000 7fccc5498ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:48:58.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.172+0000 7fccc5498ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:48:58.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.172+0000 7fccc5498ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:48:58.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.172+0000 7fccc5498ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:48:58.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.172+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.184 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.181+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:58.184 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.181+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.191+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:58.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.192+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.205 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.202+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:58.205 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.202+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.210+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:58.213 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.210+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.215+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:58.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.215+0000 7fccc5498ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:48:58.219 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.215+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.224 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.221+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:58.224 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.221+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.234 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.231+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:58.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.231+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.239+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:48:58.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.239+0000 7fccc5498ec0 20 zonegroup default 2019-12-09T13:48:58.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.239+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.244+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:58.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.244+0000 7fccc5498ec0 10 Cannot find current period zone using local zone 2019-12-09T13:48:58.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.244+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.252+0000 7fccc5498ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:48:58.255 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.252+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.267 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.264+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:48:58.267 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.264+0000 7fccc5498ec0 20 rados->read ofs=0 len=0 2019-12-09T13:48:58.276 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.273+0000 7fccc5498ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:48:58.276 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.273+0000 7fccc5498ec0 20 zone default found 2019-12-09T13:48:58.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.273+0000 7fccc5498ec0 20 started sync module instance, tier type = 2019-12-09T13:48:58.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.273+0000 7fccc5498ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:48:58.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.598+0000 7fccb26c2700 0 -- 172.21.15.174:0/416638809 >> [v2:172.21.15.174:6816/10736,v1:172.21.15.174:6817/10736] conn(0x7fcca400a550 msgr2=0x7fcca400c970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:58.607 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=0 2019-12-09T13:48:58.607 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=1 2019-12-09T13:48:58.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=2 2019-12-09T13:48:58.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=3 2019-12-09T13:48:58.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=4 2019-12-09T13:48:58.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=5 2019-12-09T13:48:58.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.604+0000 7fccc5498ec0 20 add_watcher() i=6 2019-12-09T13:48:58.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.659+0000 7fccc5498ec0 20 add_watcher() i=7 2019-12-09T13:48:58.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.659+0000 7fccc5498ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:48:58.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.660+0000 7fcc82fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:48:58.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.729+0000 7fccb26c2700 0 -- 172.21.15.174:0/1172345776 >> v2:172.21.15.174:3300/0 conn(0x55958100bdb0 msgr2=0x559580ff32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:48:58.934 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:48:58.931+0000 7fccb26c2700 0 --2- 172.21.15.174:0/1172345776 >> v2:172.21.15.174:3300/0 conn(0x55958100bdb0 0x559580ff32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T13:49:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.306+0000 7fcbf17fa700 20 reqs_thread_entry: start 2019-12-09T13:49:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.306+0000 7fcbeaffd700 20 reqs_thread_entry: start 2019-12-09T13:49:00.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.306+0000 7fccc5498ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:00.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.306+0000 7fcbe97fa700 20 reqs_thread_entry: start 2019-12-09T13:49:00.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.306+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T13:49:00.351 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.348+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:00.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.367+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:00.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:00.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1372', time=2019-12-09T14:53:50.461069+0000, chain.objs.size()=9 2019-12-09T13:49:00.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.1 2019-12-09T13:49:00.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.1_1 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.2 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.2_1 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.3 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.3_1 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.4 2019-12-09T13:49:00.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.4_1 2019-12-09T13:49:00.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.379+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~1_Pss51KXLC4xNyQZ-61DW7tvHUdAsT.5 2019-12-09T13:49:00.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.724+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:00.868 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.865+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T13:49:00.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.923+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:00.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.941+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:00.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:00.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1384', time=2019-12-09T14:54:41.850594+0000, chain.objs.size()=7 2019-12-09T13:49:00.958 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.1 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.1_1 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.1_2 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.2 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.2_1 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.2_2 2019-12-09T13:49:00.959 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:00.955+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~whrt3ODR4Kag1HzbeomRC2_UM5bDt7j.3 2019-12-09T13:49:01.142 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.139+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:01.239 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.236+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T13:49:01.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.275+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:01.293 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.290+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.306+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:01.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.306+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1120', time=2019-12-09T14:43:21.594498+0000, chain.objs.size()=2 2019-12-09T13:49:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.306+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.56__shadow_.hLk0K1r2QBXw01kAx-pZX0hFVVGG4Qw_1 2019-12-09T13:49:01.312 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.306+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.56__shadow_.hLk0K1r2QBXw01kAx-pZX0hFVVGG4Qw_2 2019-12-09T13:49:01.372 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.369+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:01.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.502+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T13:49:01.581 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.578+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:01.597 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.594+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:01.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-12-09T13:49:01.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1191', time=2019-12-09T14:44:53.769453+0000, chain.objs.size()=9 2019-12-09T13:49:01.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.1 2019-12-09T13:49:01.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.1_1 2019-12-09T13:49:01.625 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.2 2019-12-09T13:49:01.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.2_1 2019-12-09T13:49:01.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.3 2019-12-09T13:49:01.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.3_1 2019-12-09T13:49:01.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.4 2019-12-09T13:49:01.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.4_1 2019-12-09T13:49:01.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.5 2019-12-09T13:49:01.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1240', time=2019-12-09T14:49:17.984883+0000, chain.objs.size()=7 2019-12-09T13:49:01.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.1 2019-12-09T13:49:01.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.1_1 2019-12-09T13:49:01.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.622+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.1_2 2019-12-09T13:49:01.765 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.762+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.2 2019-12-09T13:49:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.817+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.2_1 2019-12-09T13:49:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.817+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.2_2 2019-12-09T13:49:01.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.817+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~JKsmT43dmn1VLivAx4VWugnZqcqEek9.3 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I', time=2019-12-09T14:57:09.406739+0000, chain.objs.size()=30 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.1 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.2 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.3 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.4 2019-12-09T13:49:01.822 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.5 2019-12-09T13:49:01.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.819+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.6 2019-12-09T13:49:01.849 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.846+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.7 2019-12-09T13:49:01.929 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.926+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.8 2019-12-09T13:49:01.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.927+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.9 2019-12-09T13:49:01.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:01.949+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.10 2019-12-09T13:49:02.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.072+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.11 2019-12-09T13:49:02.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.072+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.12 2019-12-09T13:49:02.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.13 2019-12-09T13:49:02.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.5MDseuCHIMwpmOW4knxghU0C4R62Kdv.14 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.0KvEDa_NZroEhbMaMoL9SVmY_21lGDP.15 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.7hOH4QYZc7N8vXhgo1qRvJ0_o1r5_Ob.16 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.KVNSPj2P6ugmOs94Zb4XYWxmYOfBWSS.17 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.RSoS6-6ngBbep96C88fdsOoABQigsjm.18 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.x-TeTsTe6EFeyr5UpdKgVtil0WVSn1F.19 2019-12-09T13:49:02.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.yAVLVXa4SDLFU62IcM_u4SsgE5nr_xU.20 2019-12-09T13:49:02.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.21 2019-12-09T13:49:02.157 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.152+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.22 2019-12-09T13:49:02.234 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.231+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.23 2019-12-09T13:49:02.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.367+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.24 2019-12-09T13:49:02.371 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.367+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.25 2019-12-09T13:49:02.371 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.367+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.26 2019-12-09T13:49:02.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.391+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.27 2019-12-09T13:49:02.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.391+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.28 2019-12-09T13:49:02.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.391+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.29 2019-12-09T13:49:02.404 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.402+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.78__multipart_key1.2~AgOiltvlfYbfnwaBNHTTYOXXie6N97I.30 2019-12-09T13:49:02.646 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.643+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:02.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.743+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T13:49:02.797 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.793+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:02.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.809+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:02.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.817+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:02.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.857+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T13:49:02.954 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.951+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:02.979 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.976+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:02.991 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:02.987+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:03.028 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.025+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T13:49:03.075 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.072+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:03.102 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.099+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:03.112 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.109+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:03.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.201+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T13:49:03.302 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.299+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:03.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.382+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:03.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1333', time=2019-12-09T14:51:06.932544+0000, chain.objs.size()=9 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.1 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.1_1 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.2 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.2_1 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.3 2019-12-09T13:49:03.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.3_1 2019-12-09T13:49:03.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.4 2019-12-09T13:49:03.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.4_1 2019-12-09T13:49:03.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file.mpg.2~KHbDTh_tgECj9oDHcY4QjgOxuWHFgKz.5 2019-12-09T13:49:03.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1394', time=2019-12-09T14:55:14.696780+0000, chain.objs.size()=7 2019-12-09T13:49:03.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.1 2019-12-09T13:49:03.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.1_1 2019-12-09T13:49:03.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.421+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.1_2 2019-12-09T13:49:03.678 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.675+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.2 2019-12-09T13:49:03.678 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.675+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.2_1 2019-12-09T13:49:03.678 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.675+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.2_2 2019-12-09T13:49:03.679 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.675+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~iCHzncTbV2oXYbVw1tw7oOBKm9_xxM_.3 2019-12-09T13:49:03.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.862+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:03.962 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:03.959+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T13:49:04.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.003+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.022 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.019+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:04.031 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.028+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.078 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.075+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T13:49:04.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.146+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.173 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.170+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:04.184 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.182+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.276+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T13:49:04.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.357+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.394 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.391+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1365', time=2019-12-09T14:53:04.529589+0000, chain.objs.size()=9 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.1 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.1_1 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.2 2019-12-09T13:49:04.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.2_1 2019-12-09T13:49:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.3 2019-12-09T13:49:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.3_1 2019-12-09T13:49:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.4 2019-12-09T13:49:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.409+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.4_1 2019-12-09T13:49:04.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.410+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~Xm4_Q4Bp-atyfaSxCHM0Wkx-JFgldBz.5 2019-12-09T13:49:04.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.690+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:04.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.802+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T13:49:04.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.844+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:04.866 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.863+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:04.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.878+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:04.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.878+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1379', time=2019-12-09T14:54:15.530980+0000, chain.objs.size()=9 2019-12-09T13:49:04.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.878+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.1 2019-12-09T13:49:04.882 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.878+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.1_1 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.2 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.2_1 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.3 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.3_1 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.4 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.4_1 2019-12-09T13:49:04.883 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:04.879+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~4C1FQt0Xy9dVPUf2FDFdION7TeK-HIp.5 2019-12-09T13:49:05.151 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.148+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:05.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.253+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T13:49:05.307 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.304+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:05.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.320+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1389', time=2019-12-09T14:55:09.598170+0000, chain.objs.size()=7 2019-12-09T13:49:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.1 2019-12-09T13:49:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.1_1 2019-12-09T13:49:05.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.1_2 2019-12-09T13:49:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.2 2019-12-09T13:49:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.2_1 2019-12-09T13:49:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__shadow_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.2_2 2019-12-09T13:49:05.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.330+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.75__multipart_key1.2~cgQG-b9jzjYFhHz7xeJ3g4CM99nZTX0.3 2019-12-09T13:49:05.573 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.570+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:05.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.650+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T13:49:05.694 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.691+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:05.708 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.705+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm', time=2019-12-09T14:51:54.860954+0000, chain.objs.size()=9 2019-12-09T13:49:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__multipart_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.7 2019-12-09T13:49:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__shadow_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.7_1 2019-12-09T13:49:05.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__multipart_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.8 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.720+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__shadow_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.8_1 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.721+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__multipart_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.9 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.721+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__shadow_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.9_1 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.721+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__multipart_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.10 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.721+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__shadow_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.10_1 2019-12-09T13:49:05.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:05.721+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.74__multipart_key1.2~cVwyDuKO979XUqXNo_0uJ18Lb18S4Vm.11 2019-12-09T13:49:06.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.035+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:06.146 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.143+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T13:49:06.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.192+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:06.211 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.209+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:06.220 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.217+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:06.251 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.248+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T13:49:06.296 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.293+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:06.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.308+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:06.322 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:06.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1302', time=2019-12-09T14:50:42.251939+0000, chain.objs.size()=9 2019-12-09T13:49:06.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__multipart_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.1 2019-12-09T13:49:06.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__shadow_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.1_1 2019-12-09T13:49:06.323 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__multipart_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.2 2019-12-09T13:49:06.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__shadow_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.2_1 2019-12-09T13:49:06.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__multipart_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.3 2019-12-09T13:49:06.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__shadow_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.3_1 2019-12-09T13:49:06.324 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__multipart_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.4 2019-12-09T13:49:06.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__shadow_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.4_1 2019-12-09T13:49:06.325 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.319+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.72__multipart_key1.2~t1W4hZMMmy2yoEYRSkD-4fiLijNdFnB.5 2019-12-09T13:49:06.587 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.584+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:06.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.718+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T13:49:06.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.775+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:06.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.797+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:06.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:06.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1424', time=2019-12-09T14:56:46.380071+0000, chain.objs.size()=9 2019-12-09T13:49:06.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__multipart_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.1 2019-12-09T13:49:06.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__shadow_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.1_1 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__multipart_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.2 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__shadow_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.2_1 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__multipart_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.3 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__shadow_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.3_1 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__multipart_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.4 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__shadow_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.4_1 2019-12-09T13:49:06.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:06.815+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.77__multipart_key1.2~Ta3okTDavRDXNTpjtQjb2-2dTDHrwq6.5 2019-12-09T13:49:07.088 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.085+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:07.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.161+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T13:49:07.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.232+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.251 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.248+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:07.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.258+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.291 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.288+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T13:49:07.330 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.327+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.345 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.342+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:07.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.356+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:07.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.356+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1228', time=2019-12-09T14:48:10.111778+0000, chain.objs.size()=9 2019-12-09T13:49:07.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.356+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.1 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.356+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.1_1 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.2 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.2_1 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.3 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.3_1 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.4 2019-12-09T13:49:07.361 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.4_1 2019-12-09T13:49:07.362 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.357+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~QNoKWkatHNCNQA5_jyEqks4PkD-zsVi.5 2019-12-09T13:49:07.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.566+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:07.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.708+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T13:49:07.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.758+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.776+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:07.789 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.786+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.821 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.818+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T13:49:07.889 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.886+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:07.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.914+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:07.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:07.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1409', time=2019-12-09T14:56:05.578387+0000, chain.objs.size()=9 2019-12-09T13:49:07.942 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__multipart_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.1 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__shadow_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.1_1 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__multipart_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.2 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__shadow_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.2_1 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__multipart_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.3 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__shadow_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.3_1 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__multipart_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.4 2019-12-09T13:49:07.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__shadow_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.4_1 2019-12-09T13:49:07.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:07.939+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.76__multipart_key1.2~Dh1RgBCUK5UxOh2cqmGvyXuqSGatSqC.5 2019-12-09T13:49:08.189 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.186+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:08.283 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.280+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T13:49:08.327 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.324+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:08.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.341+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:08.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:08.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1204', time=2019-12-09T14:49:20.403936+0000, chain.objs.size()=5 2019-12-09T13:49:08.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.66__shadow_.cyvFeVJ1AkqoWKMlfM4g-DWxmaCdY1G_1 2019-12-09T13:49:08.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.66__shadow_.cyvFeVJ1AkqoWKMlfM4g-DWxmaCdY1G_2 2019-12-09T13:49:08.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.66__shadow_.cyvFeVJ1AkqoWKMlfM4g-DWxmaCdY1G_3 2019-12-09T13:49:08.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.66__shadow_.cyvFeVJ1AkqoWKMlfM4g-DWxmaCdY1G_4 2019-12-09T13:49:08.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.351+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.66__shadow_.cyvFeVJ1AkqoWKMlfM4g-DWxmaCdY1G_5 2019-12-09T13:49:08.618 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.615+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:08.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.747+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T13:49:08.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.803+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:08.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.822+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1212', time=2019-12-09T14:46:46.310810+0000, chain.objs.size()=9 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.1 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.1_1 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.2 2019-12-09T13:49:08.845 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.2_1 2019-12-09T13:49:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.3 2019-12-09T13:49:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.3_1 2019-12-09T13:49:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.4 2019-12-09T13:49:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.4_1 2019-12-09T13:49:08.846 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~3TjD-9UxzzEpq3TVTn1bQXE1Tac_EW_.5 2019-12-09T13:49:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.2247', time=2019-12-09T15:48:49.976808+0000, chain.objs.size()=9 2019-12-09T13:49:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__multipart_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.1 2019-12-09T13:49:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__shadow_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.1_1 2019-12-09T13:49:08.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.841+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__multipart_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.2 2019-12-09T13:49:08.917 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.907+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__shadow_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.2_1 2019-12-09T13:49:08.941 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:08.938+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__multipart_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.3 2019-12-09T13:49:09.300 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.296+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__shadow_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.3_1 2019-12-09T13:49:09.372 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.368+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__multipart_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.4 2019-12-09T13:49:09.372 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.368+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__shadow_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.4_1 2019-12-09T13:49:09.372 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.368+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.119__multipart_key1/data/file.mpg.2~P0X-zmtO7_jklt1qonra_vYHi3iYlbi.5 2019-12-09T13:49:09.608 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.605+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:09.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.711+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T13:49:09.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.754+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:09.772 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.769+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:09.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T13:49:09.807 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1283', time=2019-12-09T14:50:09.754363+0000, chain.objs.size()=9 2019-12-09T13:49:09.807 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__multipart_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.1 2019-12-09T13:49:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__shadow_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.1_1 2019-12-09T13:49:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__multipart_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.2 2019-12-09T13:49:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__shadow_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.2_1 2019-12-09T13:49:09.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__multipart_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.3 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__shadow_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.3_1 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__multipart_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.4 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__shadow_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.4_1 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.71__multipart_key1.2~5PMld4jRKWutoJihUI3-skUCU71CrBY.5 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1334', time=2019-12-09T14:51:10.971239+0000, chain.objs.size()=9 2019-12-09T13:49:09.809 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.1 2019-12-09T13:49:09.810 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.1_1 2019-12-09T13:49:09.810 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.802+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.2 2019-12-09T13:49:09.950 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:09.947+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.2_1 2019-12-09T13:49:10.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.063+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.3 2019-12-09T13:49:10.066 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.063+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.3_1 2019-12-09T13:49:10.087 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.084+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.4 2019-12-09T13:49:10.088 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.084+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__shadow_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.4_1 2019-12-09T13:49:10.088 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.084+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.73__multipart_file2.mpg.2~42q5IFKvdYk88p9LGDY5H1m1ZbuZ6t4.5 2019-12-09T13:49:10.377 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.374+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:10.460 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.457+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T13:49:10.497 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.494+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:10.512 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.509+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:10.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-12-09T13:49:10.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~PNEj475GOkD1aut6YuZkbE_xgsktZJf', time=2019-12-09T14:42:37.413571+0000, chain.objs.size()=9 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__multipart_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.1 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__shadow_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.1_1 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__multipart_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.2 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__shadow_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.2_1 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__multipart_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.3 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__shadow_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.3_1 2019-12-09T13:49:10.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__multipart_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.4 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__shadow_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.4_1 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.52__multipart_key1.2~PNEj475GOkD1aut6YuZkbE_xgsktZJf.5 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1189', time=2019-12-09T14:45:00.463187+0000, chain.objs.size()=9 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.1 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.1_1 2019-12-09T13:49:10.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.526+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.2 2019-12-09T13:49:10.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.659+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.2_1 2019-12-09T13:49:10.731 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.728+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.3 2019-12-09T13:49:10.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.906+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.3_1 2019-12-09T13:49:10.909 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.906+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.4 2019-12-09T13:49:10.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.906+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__shadow_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.4_1 2019-12-09T13:49:10.910 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:10.906+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.64__multipart_key1.2~Tch8TcMkQAHawjm18H5INuKcXXwwN90.5 2019-12-09T13:49:11.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.271+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:11.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.353+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T13:49:11.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.394+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:11.414 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.411+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1246', time=2019-12-09T14:49:24.252202+0000, chain.objs.size()=7 2019-12-09T13:49:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.1 2019-12-09T13:49:11.428 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.1_1 2019-12-09T13:49:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.1_2 2019-12-09T13:49:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.2 2019-12-09T13:49:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.2_1 2019-12-09T13:49:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.2_2 2019-12-09T13:49:11.429 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.424+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~0tw3-IMBH7Grw1uj82zXq4hF1bkftki.3 2019-12-09T13:49:11.631 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.628+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:11.786 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.784+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T13:49:11.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.822+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:11.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.837+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:11.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.850+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:11.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.850+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.2231', time=2019-12-09T15:48:25.805048+0000, chain.objs.size()=9 2019-12-09T13:49:11.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__multipart_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.1 2019-12-09T13:49:11.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__shadow_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.1_1 2019-12-09T13:49:11.854 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__multipart_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.2 2019-12-09T13:49:11.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__shadow_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.2_1 2019-12-09T13:49:11.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__multipart_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.3 2019-12-09T13:49:11.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__shadow_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.3_1 2019-12-09T13:49:11.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__multipart_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.4 2019-12-09T13:49:11.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__shadow_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.4_1 2019-12-09T13:49:11.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:11.851+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.118__multipart_key1.2~x7oksAnWlJTKImbDjrIAWUiQxNosllL.5 2019-12-09T13:49:12.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.199+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:12.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.296+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T13:49:12.345 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.342+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:12.360 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.357+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:12.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.367+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:12.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.403+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T13:49:12.447 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.444+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:12.463 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.460+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:12.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:12.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1220', time=2019-12-09T14:47:41.302447+0000, chain.objs.size()=9 2019-12-09T13:49:12.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.1 2019-12-09T13:49:12.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.1_1 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.2 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.2_1 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.3 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.3_1 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.4 2019-12-09T13:49:12.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.4_1 2019-12-09T13:49:12.483 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.474+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~FUTj7jP4C6J_x8mDYflgansaTwBXxD3.5 2019-12-09T13:49:12.753 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.750+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:12.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.828+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T13:49:12.874 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.871+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:12.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.887+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:12.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 20 garbage collection: RGWGC::process iterating over entry tag='7b90ce1f-2e6d-4ddb-b794-a516be24e604.4385.1234', time=2019-12-09T14:48:39.186323+0000, chain.objs.size()=7 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.1 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.1_1 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.1_2 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.2 2019-12-09T13:49:12.907 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.2_1 2019-12-09T13:49:12.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__shadow_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.2_2 2019-12-09T13:49:12.908 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:12.903+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.67__multipart_key1.2~awVw_YoRzVfoK5_3fMa5kKDoNS3hq8V.3 2019-12-09T13:49:13.230 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.227+0000 7fccc5498ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-12-09T13:49:13.373 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.370+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T13:49:13.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.413+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:13.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.428+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:13.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.437+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:13.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.491+0000 7fccc5498ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T13:49:13.538 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.535+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:13.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.555+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:13.573 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.570+0000 7fccc5498ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.649+0000 7fccc5498ec0 20 remove_watcher() i=0 2019-12-09T13:49:13.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.649+0000 7fccc5498ec0 2 removed watcher, disabling cache 2019-12-09T13:49:13.717 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.714+0000 7fccc5498ec0 20 remove_watcher() i=1 2019-12-09T13:49:13.761 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.758+0000 7fccc5498ec0 20 remove_watcher() i=2 2019-12-09T13:49:13.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.830+0000 7fccc5498ec0 20 remove_watcher() i=3 2019-12-09T13:49:13.903 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.900+0000 7fccc5498ec0 20 remove_watcher() i=4 2019-12-09T13:49:13.962 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:13.959+0000 7fccc5498ec0 20 remove_watcher() i=5 2019-12-09T13:49:14.032 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.029+0000 7fccc5498ec0 20 remove_watcher() i=6 2019-12-09T13:49:14.089 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.083+0000 7fccc5498ec0 20 remove_watcher() i=7 2019-12-09T13:49:14.096 INFO:teuthology.orchestra.run.smithi174:> radosgw-admin gc process --include-all 2019-12-09T13:49:14.228 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.225+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.287+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.287+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.301 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.298+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.301 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.298+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.316 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.313+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.316 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.313+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.326+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:14.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.477+0000 7fa176906ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:14.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.477+0000 7fa176906ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:14.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.477+0000 7fa176906ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:14.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.477+0000 7fa176906ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:14.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.477+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.490 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.487+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:14.490 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.487+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.498+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:14.501 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.498+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.508+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:14.511 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.508+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.516+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:14.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.516+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.525 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.523+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.523+0000 7fa176906ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:14.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.523+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.528+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.531 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.528+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.539+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:14.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.539+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.553 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.550+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:14.554 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.550+0000 7fa176906ec0 20 zonegroup default 2019-12-09T13:49:14.554 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.550+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.559+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.559+0000 7fa176906ec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:14.562 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.559+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.566+0000 7fa176906ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:14.569 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.566+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.581 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.578+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:14.581 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.578+0000 7fa176906ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:14.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.586+0000 7fa176906ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:14.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.586+0000 7fa176906ec0 20 zone default found 2019-12-09T13:49:14.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.586+0000 7fa176906ec0 20 started sync module instance, tier type = 2019-12-09T13:49:14.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.586+0000 7fa176906ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:14.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=0 2019-12-09T13:49:14.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=1 2019-12-09T13:49:14.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=2 2019-12-09T13:49:14.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=3 2019-12-09T13:49:14.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=4 2019-12-09T13:49:14.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.921+0000 7fa176906ec0 20 add_watcher() i=5 2019-12-09T13:49:14.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.928+0000 7fa176906ec0 20 add_watcher() i=6 2019-12-09T13:49:14.954 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.951+0000 7fa176906ec0 20 add_watcher() i=7 2019-12-09T13:49:14.954 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.951+0000 7fa176906ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:14.955 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.951+0000 7fa0b7fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:14.957 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:14.954+0000 7fa163b30700 0 -- 172.21.15.174:0/2504988692 >> v2:172.21.15.106:3300/0 conn(0x55d1d2966610 msgr2=0x55d1d2961ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:49:15.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:15.003+0000 7fa163b30700 0 -- 172.21.15.174:0/338275135 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7fa0a4022480 msgr2=0x7fa0a4024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:16.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.564+0000 7fa0a27fc700 20 reqs_thread_entry: start 2019-12-09T13:49:16.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.564+0000 7fa09bfff700 20 reqs_thread_entry: start 2019-12-09T13:49:16.567 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.564+0000 7fa176906ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:16.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.565+0000 7fa09a7fc700 20 reqs_thread_entry: start 2019-12-09T13:49:16.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.565+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-12-09T13:49:16.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.640+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:16.670 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.667+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:16.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.679+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:16.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.724+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-12-09T13:49:16.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.811+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:16.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.836+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:16.847 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.844+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:16.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.911+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-12-09T13:49:16.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.959+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:16.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.976+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:16.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:16.987+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.023 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.020+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-12-09T13:49:17.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.064+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.082+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.095 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.091+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.175 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.172+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-12-09T13:49:17.234 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.231+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.263 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.260+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.280 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.277+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.317+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-12-09T13:49:17.367 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.364+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.383 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.380+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.396 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.393+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.435 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.432+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-12-09T13:49:17.486 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.483+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.503+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.517 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.514+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.565+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-12-09T13:49:17.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.648+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.689+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.700+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.765 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.755+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-12-09T13:49:17.807 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.804+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.820+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.831 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.829+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.862+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-12-09T13:49:17.916 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.913+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.932+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:17.947 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.944+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:17.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:17.983+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-12-09T13:49:18.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.042+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.076 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.072+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.088 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.085+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.139 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.136+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-12-09T13:49:18.202 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.199+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.222 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.219+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.235 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.232+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.307+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-12-09T13:49:18.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.385+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.404+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.415+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.455 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.452+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-12-09T13:49:18.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.527+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.555 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.552+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.568 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.565+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.619 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.616+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-12-09T13:49:18.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.689+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.716+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.733+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.777+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-12-09T13:49:18.827 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.824+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.844 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.841+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.850+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.890 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.887+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-12-09T13:49:18.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.934+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.949+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:18.963 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.960+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:18.999 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:18.996+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-12-09T13:49:19.044 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.041+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.060 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.057+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.069 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.066+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.098+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-12-09T13:49:19.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.142+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.162 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.159+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.172 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.168+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.206 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.203+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-12-09T13:49:19.292 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.289+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.314 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.311+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.321+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.377 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.374+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-12-09T13:49:19.437 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.434+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.456 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.453+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.466+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.510+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-12-09T13:49:19.590 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.587+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.613 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.610+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.626 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.623+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.679+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-12-09T13:49:19.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.742+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.765+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.779 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.777+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.825+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-12-09T13:49:19.886 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.883+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.920 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.917+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:19.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.927+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:19.977 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:19.968+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-12-09T13:49:20.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.024+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.048+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.067 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.064+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.118 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.115+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-12-09T13:49:20.170 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.167+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.192+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.208 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.205+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.274+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-12-09T13:49:20.348 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.345+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.369 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.366+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.382+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.431 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.428+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-12-09T13:49:20.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.482+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.506 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.503+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.516+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.555 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.552+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-12-09T13:49:20.613 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.610+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.632 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.629+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.645 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.638+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.681+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-12-09T13:49:20.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.735+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.755 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.752+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.763+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.804 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.801+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-12-09T13:49:20.853 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.850+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.870 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.866+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:20.877 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.874+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:20.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:20.919+0000 7fa176906ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-12-09T13:49:21.016 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.013+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:21.046 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.042+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-12-09T13:49:21.062 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.059+0000 7fa176906ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-12-09T13:49:21.173 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.170+0000 7fa176906ec0 20 remove_watcher() i=0 2019-12-09T13:49:21.173 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.170+0000 7fa176906ec0 2 removed watcher, disabling cache 2019-12-09T13:49:21.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.223+0000 7fa176906ec0 20 remove_watcher() i=1 2019-12-09T13:49:21.274 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.271+0000 7fa176906ec0 20 remove_watcher() i=2 2019-12-09T13:49:21.317 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.314+0000 7fa176906ec0 20 remove_watcher() i=3 2019-12-09T13:49:21.368 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.365+0000 7fa176906ec0 20 remove_watcher() i=4 2019-12-09T13:49:21.406 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.403+0000 7fa176906ec0 20 remove_watcher() i=5 2019-12-09T13:49:21.471 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.467+0000 7fa176906ec0 20 remove_watcher() i=6 2019-12-09T13:49:21.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:21.523+0000 7fa176906ec0 20 remove_watcher() i=7 2019-12-09T13:49:21.536 INFO:teuthology.run_tasks:Running task s3tests... 2019-12-09T13:49:21.551 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-12-09T13:49:21.551 INFO:tasks.s3tests:Downloading s3-tests... 2019-12-09T13:49:21.551 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-12-09T13:49:21.551 INFO:teuthology.orchestra.run.smithi174:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-12-09T13:49:21.644 INFO:teuthology.orchestra.run.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-12-09T13:49:22.087 INFO:tasks.s3tests:Creating rgw users... 2019-12-09T13:49:22.088 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-12-09T13:49:22.088 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key TAZOQBUSHGUKLFCLQYQR --secret TkRwYyzhkxIOKwkwF5PTi3j/IBAixA8C6A8HPTJI3YCos3H8s0pEnQ== --email bar.client.0+test@test.test --cluster ceph 2019-12-09T13:49:22.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.215+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.252+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.256 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.252+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.264 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.261+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.264 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.261+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.269+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.273 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.269+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.291+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:22.492 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.489+0000 7f0fcc93dec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:22.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.489+0000 7f0fcc93dec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:22.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.489+0000 7f0fcc93dec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:22.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.489+0000 7f0fcc93dec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:22.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.489+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.502+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:22.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.502+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.514 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.511+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:22.515 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.511+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.525+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:22.529 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.525+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.538 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.534+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:22.539 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.534+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.541+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.541+0000 7f0fcc93dec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:22.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.541+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.547+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.547+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.563+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:22.566 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.563+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.572+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:22.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.572+0000 7f0fcc93dec0 20 zonegroup default 2019-12-09T13:49:22.575 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.572+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.581 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.578+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.578+0000 7f0fcc93dec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:22.582 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.578+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.588 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.585+0000 7f0fcc93dec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:22.589 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.585+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.600+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:22.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.600+0000 7f0fcc93dec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:22.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.610+0000 7f0fcc93dec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:22.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.610+0000 7f0fcc93dec0 20 zone default found 2019-12-09T13:49:22.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.610+0000 7f0fcc93dec0 20 started sync module instance, tier type = 2019-12-09T13:49:22.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.610+0000 7f0fcc93dec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:22.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=0 2019-12-09T13:49:22.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=1 2019-12-09T13:49:22.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=2 2019-12-09T13:49:22.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=3 2019-12-09T13:49:22.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=4 2019-12-09T13:49:22.971 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.967+0000 7f0fcc93dec0 20 add_watcher() i=5 2019-12-09T13:49:22.978 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:22.975+0000 7f0fcc93dec0 20 add_watcher() i=6 2019-12-09T13:49:23.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:23.002+0000 7f0fcc93dec0 20 add_watcher() i=7 2019-12-09T13:49:23.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:23.002+0000 7f0fcc93dec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:23.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:23.002+0000 7f0f8a7dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:23.009 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:23.006+0000 7f0fb1366700 0 -- 172.21.15.174:0/525688662 >> v1:172.21.15.106:6789/0 conn(0x55e2cc064540 legacy=0x55e2cc066a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:23.051 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:23.048+0000 7f0fb9b67700 0 -- 172.21.15.174:0/100414858 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x55e2cc076970 msgr2=0x55e2cc078d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:24.804 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.799+0000 7f0f00ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.800+0000 7f0efa7fc700 20 reqs_thread_entry: start 2019-12-09T13:49:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.800+0000 7f0fcc93dec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.801+0000 7f0ef8ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:24.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.801+0000 7f0fcc93dec0 20 get_system_obj_state: rctx=0x7ffd97bfb068 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55e2cc0e0180 s->prefetch_data=0 2019-12-09T13:49:24.806 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.801+0000 7f0fcc93dec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:49:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.810+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T13:49:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.810+0000 7f0fcc93dec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:24.814 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.810+0000 7f0fcc93dec0 20 get_system_obj_state: rctx=0x7ffd97bfb078 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55e2cc0e1f10 s->prefetch_data=0 2019-12-09T13:49:24.815 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.810+0000 7f0fcc93dec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T13:49:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.815+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T13:49:24.819 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.815+0000 7f0fcc93dec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:49:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.815+0000 7f0fcc93dec0 20 get_system_obj_state: rctx=0x7ffd97bfb078 obj=default.rgw.meta:users.keys:TAZOQBUSHGUKLFCLQYQR state=0x55e2cc0e1f10 s->prefetch_data=0 2019-12-09T13:49:24.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.815+0000 7f0fcc93dec0 10 cache get: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR : miss 2019-12-09T13:49:24.825 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.821+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR info.flags=0x0 2019-12-09T13:49:24.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.821+0000 7f0fcc93dec0 10 adding default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR to cache LRU end 2019-12-09T13:49:24.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.821+0000 7f0fcc93dec0 20 get_system_obj_state: rctx=0x7ffd97bfac88 obj=default.rgw.meta:users.keys:TAZOQBUSHGUKLFCLQYQR state=0x55e2cc0e1f10 s->prefetch_data=0 2019-12-09T13:49:24.826 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.821+0000 7f0fcc93dec0 10 cache get: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.829+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR info.flags=0x0 2019-12-09T13:49:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.829+0000 7f0fcc93dec0 10 moving default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR to cache LRU end 2019-12-09T13:49:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.829+0000 7f0fcc93dec0 20 get_system_obj_state: rctx=0x7ffd97bfae08 obj=default.rgw.meta:users.keys:TAZOQBUSHGUKLFCLQYQR state=0x55e2cc0e1f10 s->prefetch_data=0 2019-12-09T13:49:24.833 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.829+0000 7f0fcc93dec0 10 cache get: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:24.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.834+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR info.flags=0x0 2019-12-09T13:49:24.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.834+0000 7f0fcc93dec0 10 moving default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR to cache LRU end 2019-12-09T13:49:24.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.922+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:49:24.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.922+0000 7f0fcc93dec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:24.927 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.922+0000 7f0fcc93dec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-12-09T13:49:24.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.935+0000 7f0f9affd700 10 RGWWatcher::handle_notify() notify_id 579820585023 cookie 94432573844432 notifier 4640 bl.length()=617 2019-12-09T13:49:24.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.935+0000 7f0f9affd700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:49:24.939 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:24.935+0000 7f0f9affd700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:26.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.191+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:26.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.191+0000 7f0fcc93dec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:49:26.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.191+0000 7f0fcc93dec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T13:49:26.214 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.210+0000 7f0f9affd700 10 RGWWatcher::handle_notify() notify_id 579820585024 cookie 94432573844432 notifier 4640 bl.length()=194 2019-12-09T13:49:26.214 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.210+0000 7f0f9affd700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:26.214 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.210+0000 7f0f9affd700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T13:49:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.638+0000 7f0fcc93dec0 10 cache put: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR info.flags=0x7 2019-12-09T13:49:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.638+0000 7f0fcc93dec0 10 moving default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR to cache LRU end 2019-12-09T13:49:26.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.638+0000 7f0fcc93dec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=186 2019-12-09T13:49:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.653+0000 7f0f9affd700 10 RGWWatcher::handle_notify() notify_id 579820585025 cookie 94432573854944 notifier 4640 bl.length()=186 2019-12-09T13:49:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.653+0000 7f0f9affd700 10 cache put: name=default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR info.flags=0x7 2019-12-09T13:49:26.657 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.653+0000 7f0f9affd700 10 moving default.rgw.meta+users.keys+TAZOQBUSHGUKLFCLQYQR to cache LRU end 2019-12-09T13:49:26.700 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "bar.client.0", 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. bar.client.0", 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T13:49:26.701 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "bar.client.0", 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "TAZOQBUSHGUKLFCLQYQR", 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "TkRwYyzhkxIOKwkwF5PTi3j/IBAixA8C6A8HPTJI3YCos3H8s0pEnQ==" 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T13:49:26.702 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T13:49:26.703 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:26.704 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:26.705 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T13:49:26.706 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:26.742 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.738+0000 7f0fcc93dec0 20 remove_watcher() i=0 2019-12-09T13:49:26.742 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.738+0000 7f0fcc93dec0 2 removed watcher, disabling cache 2019-12-09T13:49:26.850 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.846+0000 7f0fcc93dec0 20 remove_watcher() i=1 2019-12-09T13:49:26.895 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.891+0000 7f0fcc93dec0 20 remove_watcher() i=2 2019-12-09T13:49:26.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:26.939+0000 7f0fcc93dec0 20 remove_watcher() i=3 2019-12-09T13:49:27.006 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.002+0000 7f0fcc93dec0 20 remove_watcher() i=4 2019-12-09T13:49:27.073 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.069+0000 7f0fcc93dec0 20 remove_watcher() i=5 2019-12-09T13:49:27.170 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.166+0000 7f0fcc93dec0 20 remove_watcher() i=6 2019-12-09T13:49:27.215 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.211+0000 7f0fcc93dec0 20 remove_watcher() i=7 2019-12-09T13:49:27.223 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 5307910756 --totp-seed BEPVWP6Z4EST7DA6RUZCLHKPIR77ACEUH5VAZN6V6HYOFUXDRBTKI5UW5FXUUTKV --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T13:49:27.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.366+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.402+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.407 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.402+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.412+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.412+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.420+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.426 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.420+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.448 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.444+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:27.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.639+0000 7f9f0dbbeec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.639+0000 7f9f0dbbeec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.639+0000 7f9f0dbbeec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.639+0000 7f9f0dbbeec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.639+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.657+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:27.661 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.657+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.676 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.669+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:27.676 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.669+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.688 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.684+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:27.688 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.684+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.700+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:27.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.700+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.706+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.706+0000 7f9f0dbbeec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:27.711 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.706+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.714+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.720 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.714+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.732 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.728+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:27.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.728+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.744 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.740+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:27.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.740+0000 7f9f0dbbeec0 20 zonegroup default 2019-12-09T13:49:27.745 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.740+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.747+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.751 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.747+0000 7f9f0dbbeec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:27.751 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.747+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.756 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.752+0000 7f9f0dbbeec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:27.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.753+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.772 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.767+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:27.773 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.767+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:27.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.779+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:27.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.779+0000 7f9f0dbbeec0 20 zone default found 2019-12-09T13:49:27.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.779+0000 7f9f0dbbeec0 20 started sync module instance, tier type = 2019-12-09T13:49:27.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:27.779+0000 7f9f0dbbeec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:28.154 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=0 2019-12-09T13:49:28.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=1 2019-12-09T13:49:28.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=2 2019-12-09T13:49:28.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=3 2019-12-09T13:49:28.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=4 2019-12-09T13:49:28.155 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=5 2019-12-09T13:49:28.156 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.151+0000 7f9f0dbbeec0 20 add_watcher() i=6 2019-12-09T13:49:28.197 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.193+0000 7f9f0dbbeec0 20 add_watcher() i=7 2019-12-09T13:49:28.198 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.193+0000 7f9f0dbbeec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:28.198 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.194+0000 7f9eca7ec700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:28.208 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.204+0000 7f9efa5e7700 0 -- 172.21.15.174:0/2563097013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca46cef540 msgr2=0x55ca46cf1a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:28.230 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:28.226+0000 7f9efa5e7700 0 -- 172.21.15.174:0/3798767690 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x7f9e3c003a70 msgr2=0x7f9e3c003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T13:49:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9e40ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9e3a7fc700 20 reqs_thread_entry: start 2019-12-09T13:49:29.627 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9f0dbbeec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:29.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9e38ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:29.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9f0dbbeec0 20 get_system_obj_state: rctx=0x7ffffc89f0f8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55ca46d61d70 s->prefetch_data=0 2019-12-09T13:49:29.628 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.623+0000 7f9f0dbbeec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:49:29.641 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.636+0000 7f9f0dbbeec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:29.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.651+0000 7f9f0dbbeec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T13:49:29.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.651+0000 7f9f0dbbeec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T13:49:29.656 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:29.651+0000 7f9f0dbbeec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.910+0000 7f9f0dbbeec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:49:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.910+0000 7f9f0dbbeec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:31.914 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.910+0000 7f9f0dbbeec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-12-09T13:49:31.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.921+0000 7f9eebfff700 10 RGWWatcher::handle_notify() notify_id 588410519618 cookie 94327259634704 notifier 4649 bl.length()=631 2019-12-09T13:49:31.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.921+0000 7f9eebfff700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T13:49:31.925 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:31.921+0000 7f9eebfff700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T13:49:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.234+0000 7f9f0dbbeec0 20 remove_watcher() i=0 2019-12-09T13:49:32.240 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.234+0000 7f9f0dbbeec0 2 removed watcher, disabling cache 2019-12-09T13:49:32.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.294+0000 7f9f0dbbeec0 20 remove_watcher() i=1 2019-12-09T13:49:32.341 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.337+0000 7f9f0dbbeec0 20 remove_watcher() i=2 2019-12-09T13:49:32.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.393+0000 7f9f0dbbeec0 20 remove_watcher() i=3 2019-12-09T13:49:32.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.445+0000 7f9f0dbbeec0 20 remove_watcher() i=4 2019-12-09T13:49:32.837 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:32.833+0000 7f9f0dbbeec0 20 remove_watcher() i=5 2019-12-09T13:49:33.290 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.287+0000 7f9f0dbbeec0 20 remove_watcher() i=6 2019-12-09T13:49:33.337 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.333+0000 7f9f0dbbeec0 20 remove_watcher() i=7 2019-12-09T13:49:33.345 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-12-09T13:49:33.345 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key JBMMFVPQAOCYGFILKQBF --secret cVgvEj967wL047KUG+BgqbUdmr5MPZzdQzSp0PnO1pLyip389y3C0Q== --email foo.client.0+test@test.test --cluster ceph 2019-12-09T13:49:33.478 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.474+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.495+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.499 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.495+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.500+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.504 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.500+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.509 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.505+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.510 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.505+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.520 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.516+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:33.667 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.663+0000 7eff24726ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:33.667 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.663+0000 7eff24726ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:33.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.663+0000 7eff24726ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:33.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.663+0000 7eff24726ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:33.668 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.663+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.678 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.672+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:33.678 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.673+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.681+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:33.685 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.681+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.699 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.695+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:33.699 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.695+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.708 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.704+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:33.708 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.704+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.708+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.712 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.708+0000 7eff24726ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:33.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.708+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.714+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.714+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.727 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.723+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:33.728 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.723+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.739 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.735+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:33.739 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.735+0000 7eff24726ec0 20 zonegroup default 2019-12-09T13:49:33.740 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.735+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.746+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.746+0000 7eff24726ec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:33.750 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.746+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.753+0000 7eff24726ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:33.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.753+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.781 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.763+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.763+0000 7eff24726ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.773+0000 7eff24726ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.773+0000 7eff24726ec0 20 zone default found 2019-12-09T13:49:33.782 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.773+0000 7eff24726ec0 20 started sync module instance, tier type = 2019-12-09T13:49:33.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:33.773+0000 7eff24726ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:34.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=0 2019-12-09T13:49:34.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=1 2019-12-09T13:49:34.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=2 2019-12-09T13:49:34.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=3 2019-12-09T13:49:34.124 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=4 2019-12-09T13:49:34.125 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=5 2019-12-09T13:49:34.125 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.120+0000 7eff24726ec0 20 add_watcher() i=6 2019-12-09T13:49:34.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.148+0000 7eff24726ec0 20 add_watcher() i=7 2019-12-09T13:49:34.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.148+0000 7eff24726ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:34.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.148+0000 7efee27dc700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:34.167 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.154+0000 7eff1114f700 0 -- 172.21.15.174:0/2760631461 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x556f9bb52280 msgr2=0x556f9bb54790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:49:34.181 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:34.177+0000 7eff1114f700 0 -- 172.21.15.174:0/1038064485 >> v2:172.21.15.106:3300/0 conn(0x556f9bb59700 msgr2=0x556f9bb5bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:35.523 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.518+0000 7efe58ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:35.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.519+0000 7efe527fc700 20 reqs_thread_entry: start 2019-12-09T13:49:35.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.519+0000 7eff24726ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:35.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.519+0000 7efe50ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:35.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.519+0000 7eff24726ec0 20 get_system_obj_state: rctx=0x7ffe8cdd2878 obj=default.rgw.meta:users.uid:foo.client.0 state=0x556f9bbcdf90 s->prefetch_data=0 2019-12-09T13:49:35.524 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.519+0000 7eff24726ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:49:35.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.525+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T13:49:35.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.525+0000 7eff24726ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:35.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.525+0000 7eff24726ec0 20 get_system_obj_state: rctx=0x7ffe8cdd2888 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x556f9bbcfd20 s->prefetch_data=0 2019-12-09T13:49:35.530 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.525+0000 7eff24726ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T13:49:35.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.531+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T13:49:35.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.531+0000 7eff24726ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:49:35.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.531+0000 7eff24726ec0 20 get_system_obj_state: rctx=0x7ffe8cdd2888 obj=default.rgw.meta:users.keys:JBMMFVPQAOCYGFILKQBF state=0x556f9bbcfd20 s->prefetch_data=0 2019-12-09T13:49:35.536 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.531+0000 7eff24726ec0 10 cache get: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF : miss 2019-12-09T13:49:35.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.536+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF info.flags=0x0 2019-12-09T13:49:35.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.536+0000 7eff24726ec0 10 adding default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF to cache LRU end 2019-12-09T13:49:35.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.536+0000 7eff24726ec0 20 get_system_obj_state: rctx=0x7ffe8cdd2498 obj=default.rgw.meta:users.keys:JBMMFVPQAOCYGFILKQBF state=0x556f9bbcfd20 s->prefetch_data=0 2019-12-09T13:49:35.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.536+0000 7eff24726ec0 10 cache get: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:35.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.541+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF info.flags=0x0 2019-12-09T13:49:35.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.541+0000 7eff24726ec0 10 moving default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF to cache LRU end 2019-12-09T13:49:35.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.541+0000 7eff24726ec0 20 get_system_obj_state: rctx=0x7ffe8cdd2618 obj=default.rgw.meta:users.keys:JBMMFVPQAOCYGFILKQBF state=0x556f9bbcfd20 s->prefetch_data=0 2019-12-09T13:49:35.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.541+0000 7eff24726ec0 10 cache get: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:35.551 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.546+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF info.flags=0x0 2019-12-09T13:49:35.552 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.546+0000 7eff24726ec0 10 moving default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF to cache LRU end 2019-12-09T13:49:35.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.611+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:49:35.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.611+0000 7eff24726ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:35.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.611+0000 7eff24726ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-12-09T13:49:35.639 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.632+0000 7efef2ffd700 10 RGWWatcher::handle_notify() notify_id 592705486911 cookie 93937841978384 notifier 4960 bl.length()=617 2019-12-09T13:49:35.639 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.632+0000 7efef2ffd700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:49:35.640 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:35.632+0000 7efef2ffd700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:36.296 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.292+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:36.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.292+0000 7eff24726ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:49:36.297 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.292+0000 7eff24726ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T13:49:36.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.305+0000 7efef2ffd700 10 RGWWatcher::handle_notify() notify_id 592705486876 cookie 93937841994224 notifier 4960 bl.length()=194 2019-12-09T13:49:36.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.305+0000 7efef2ffd700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:36.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.305+0000 7efef2ffd700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T13:49:36.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.595+0000 7eff24726ec0 10 cache put: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF info.flags=0x7 2019-12-09T13:49:36.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.595+0000 7eff24726ec0 10 moving default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF to cache LRU end 2019-12-09T13:49:36.599 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.595+0000 7eff24726ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=186 2019-12-09T13:49:36.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.610+0000 7efef2ffd700 10 RGWWatcher::handle_notify() notify_id 592705486911 cookie 93937841983616 notifier 4960 bl.length()=186 2019-12-09T13:49:36.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.610+0000 7efef2ffd700 10 cache put: name=default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF info.flags=0x7 2019-12-09T13:49:36.616 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.610+0000 7efef2ffd700 10 moving default.rgw.meta+users.keys+JBMMFVPQAOCYGFILKQBF to cache LRU end 2019-12-09T13:49:36.685 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T13:49:36.685 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "foo.client.0", 2019-12-09T13:49:36.685 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. foo.client.0", 2019-12-09T13:49:36.685 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T13:49:36.686 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "foo.client.0", 2019-12-09T13:49:36.687 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "JBMMFVPQAOCYGFILKQBF", 2019-12-09T13:49:36.687 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "cVgvEj967wL047KUG+BgqbUdmr5MPZzdQzSp0PnO1pLyip389y3C0Q==" 2019-12-09T13:49:36.687 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T13:49:36.687 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T13:49:36.687 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T13:49:36.688 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T13:49:36.688 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T13:49:36.688 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T13:49:36.688 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T13:49:36.688 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:36.689 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:36.690 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:36.690 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T13:49:36.690 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:36.690 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T13:49:36.691 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T13:49:36.692 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T13:49:36.692 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:36.756 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.752+0000 7eff24726ec0 20 remove_watcher() i=0 2019-12-09T13:49:36.757 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.752+0000 7eff24726ec0 2 removed watcher, disabling cache 2019-12-09T13:49:36.818 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.814+0000 7eff24726ec0 20 remove_watcher() i=1 2019-12-09T13:49:36.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.875+0000 7eff24726ec0 20 remove_watcher() i=2 2019-12-09T13:49:36.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.924+0000 7eff24726ec0 20 remove_watcher() i=3 2019-12-09T13:49:36.975 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:36.971+0000 7eff24726ec0 20 remove_watcher() i=4 2019-12-09T13:49:37.028 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.024+0000 7eff24726ec0 20 remove_watcher() i=5 2019-12-09T13:49:37.072 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.068+0000 7eff24726ec0 20 remove_watcher() i=6 2019-12-09T13:49:37.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.098+0000 7eff24726ec0 20 remove_watcher() i=7 2019-12-09T13:49:37.144 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 2893675690 --totp-seed BBCSUTOT7VMLZPEYT6QRFVQDWJC4BUMMZIJSWGWZ5UP6PGGOCNCECGN3M64MHYJO --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T13:49:37.281 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.277+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.307+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.312 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.307+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.315+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.315+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.325+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.325+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.347 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.343+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.540+0000 7f7aab16aec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:37.544 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.540+0000 7f7aab16aec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.540+0000 7f7aab16aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.540+0000 7f7aab16aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:37.545 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.540+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.553+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:37.557 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.553+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.565 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.560+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:37.565 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.560+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.576 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.572+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:37.576 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.572+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.592 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.588+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:37.593 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.588+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.596+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.596+0000 7f7aab16aec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:37.601 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.596+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.609 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.605+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.610 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.605+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.621 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.617+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:37.622 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.618+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.625+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:37.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.625+0000 7f7aab16aec0 20 zonegroup default 2019-12-09T13:49:37.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.625+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.632+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.636 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.632+0000 7f7aab16aec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:37.637 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.632+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.638+0000 7f7aab16aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:37.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.638+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.649+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:37.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.649+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:37.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.658+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:37.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.658+0000 7f7aab16aec0 20 zone default found 2019-12-09T13:49:37.662 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.658+0000 7f7aab16aec0 20 started sync module instance, tier type = 2019-12-09T13:49:37.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:37.658+0000 7f7aab16aec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:38.011 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.007+0000 7f7aab16aec0 20 add_watcher() i=0 2019-12-09T13:49:38.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.008+0000 7f7aab16aec0 20 add_watcher() i=1 2019-12-09T13:49:38.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.008+0000 7f7aab16aec0 20 add_watcher() i=2 2019-12-09T13:49:38.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.008+0000 7f7aab16aec0 20 add_watcher() i=3 2019-12-09T13:49:38.012 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.008+0000 7f7aab16aec0 20 add_watcher() i=4 2019-12-09T13:49:38.013 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.008+0000 7f7aab16aec0 20 add_watcher() i=5 2019-12-09T13:49:38.014 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.010+0000 7f7aab16aec0 20 add_watcher() i=6 2019-12-09T13:49:38.042 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.037+0000 7f7aab16aec0 20 add_watcher() i=7 2019-12-09T13:49:38.042 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.037+0000 7f7aab16aec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:38.042 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.037+0000 7f7a68fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:38.045 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.041+0000 7f7a97b93700 0 -- 172.21.15.174:0/97796152 >> v2:172.21.15.106:3300/0 conn(0x5572db12a3f0 msgr2=0x5572db12c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:49:38.087 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:38.083+0000 7f7a98394700 0 -- 172.21.15.174:0/2926848270 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x5572db13b610 msgr2=0x5572db13da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:39.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.709+0000 7f79daffd700 20 reqs_thread_entry: start 2019-12-09T13:49:39.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.710+0000 7f79d8ff9700 20 reqs_thread_entry: start 2019-12-09T13:49:39.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.710+0000 7f7aab16aec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:39.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.710+0000 7f79ceffd700 20 reqs_thread_entry: start 2019-12-09T13:49:39.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.710+0000 7f7aab16aec0 20 get_system_obj_state: rctx=0x7ffe3ce55708 obj=default.rgw.meta:users.uid:foo.client.0 state=0x5572db1a4d10 s->prefetch_data=0 2019-12-09T13:49:39.715 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.710+0000 7f7aab16aec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T13:49:39.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T13:49:39.725 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:39.726 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:49:39.726 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:49:39.726 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:49:39.726 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.721+0000 7f7aab16aec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:39.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.734+0000 7f7aab16aec0 20 rados_obj.operate() r=0 bl.length=432 2019-12-09T13:49:39.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.734+0000 7f7aab16aec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T13:49:39.738 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.734+0000 7f7aab16aec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:39.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.908+0000 7f7aab16aec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:49:39.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.908+0000 7f7aab16aec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:39.912 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.908+0000 7f7aab16aec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-12-09T13:49:39.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.920+0000 7f7a7dffb700 10 RGWWatcher::handle_notify() notify_id 592705486912 cookie 93951789968096 notifier 4975 bl.length()=631 2019-12-09T13:49:39.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.921+0000 7f7a7dffb700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T13:49:39.926 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:39.921+0000 7f7a7dffb700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T13:49:40.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.481+0000 7f7aab16aec0 20 remove_watcher() i=0 2019-12-09T13:49:40.485 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.481+0000 7f7aab16aec0 2 removed watcher, disabling cache 2019-12-09T13:49:40.546 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.542+0000 7f7aab16aec0 20 remove_watcher() i=1 2019-12-09T13:49:40.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.598+0000 7f7aab16aec0 20 remove_watcher() i=2 2019-12-09T13:49:40.651 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.647+0000 7f7aab16aec0 20 remove_watcher() i=3 2019-12-09T13:49:40.699 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.695+0000 7f7aab16aec0 20 remove_watcher() i=4 2019-12-09T13:49:40.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.742+0000 7f7aab16aec0 20 remove_watcher() i=5 2019-12-09T13:49:40.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.804+0000 7f7aab16aec0 20 remove_watcher() i=6 2019-12-09T13:49:40.861 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.856+0000 7f7aab16aec0 20 remove_watcher() i=7 2019-12-09T13:49:40.870 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-12-09T13:49:40.870 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key NLHORHQOCYKKTUEZLGGT --secret 0cdUYuWXWVo1bMGbtHyEamfj2SOLLZXzxYhmbOnmWarGT6JiF5+HHg== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-12-09T13:49:41.005 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:40.995+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.037 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.032+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.038 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.032+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.047 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.043+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.047 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.043+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.058 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.054+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.059 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.054+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.079 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.076+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:41.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.274+0000 7f90f7f39ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:41.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.274+0000 7f90f7f39ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:41.278 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.274+0000 7f90f7f39ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:41.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.274+0000 7f90f7f39ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:41.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.274+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.293+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:41.298 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.293+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.304+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:41.309 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.304+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.319 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.315+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:41.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.315+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.330+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:41.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.330+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.344 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.340+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.344 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.340+0000 7f90f7f39ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:41.344 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.340+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.351+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.356 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.351+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.374+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:41.378 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.374+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.391 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.386+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:41.391 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.386+0000 7f90f7f39ec0 20 zonegroup default 2019-12-09T13:49:41.391 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.386+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.393+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.393+0000 7f90f7f39ec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:41.398 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.393+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.403 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.399+0000 7f90f7f39ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:41.404 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.399+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.408+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:41.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.408+0000 7f90f7f39ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:41.421 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.417+0000 7f90f7f39ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:41.421 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.417+0000 7f90f7f39ec0 20 zone default found 2019-12-09T13:49:41.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.417+0000 7f90f7f39ec0 20 started sync module instance, tier type = 2019-12-09T13:49:41.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.417+0000 7f90f7f39ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:41.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=0 2019-12-09T13:49:41.838 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=1 2019-12-09T13:49:41.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=2 2019-12-09T13:49:41.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=3 2019-12-09T13:49:41.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=4 2019-12-09T13:49:41.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=5 2019-12-09T13:49:41.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.834+0000 7f90f7f39ec0 20 add_watcher() i=6 2019-12-09T13:49:41.879 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.874+0000 7f90f7f39ec0 20 add_watcher() i=7 2019-12-09T13:49:41.880 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.874+0000 7f90f7f39ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:41.880 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.874+0000 7f90b5fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:41.885 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.881+0000 7f90e4962700 0 -- 172.21.15.174:0/1708511690 >> v2:172.21.15.174:3300/0 conn(0x55a5174475e0 msgr2=0x55a517449af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T13:49:41.930 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:41.926+0000 7f90e5964700 0 -- 172.21.15.174:0/3370249574 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f9028022490 msgr2=0x7f9028024960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T13:49:43.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.298+0000 7f9027fff700 20 reqs_thread_entry: start 2019-12-09T13:49:43.303 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.299+0000 7f9025ffb700 20 reqs_thread_entry: start 2019-12-09T13:49:43.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.299+0000 7f90f7f39ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:43.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.299+0000 7f9013fff700 20 reqs_thread_entry: start 2019-12-09T13:49:43.304 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.300+0000 7f90f7f39ec0 20 get_system_obj_state: rctx=0x7fffb1b67918 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55a5174c2090 s->prefetch_data=0 2019-12-09T13:49:43.305 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.300+0000 7f90f7f39ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T13:49:43.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.306+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-12-09T13:49:43.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.306+0000 7f90f7f39ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:43.310 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.306+0000 7f90f7f39ec0 20 get_system_obj_state: rctx=0x7fffb1b67928 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x55a5174c3e40 s->prefetch_data=0 2019-12-09T13:49:43.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.306+0000 7f90f7f39ec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-12-09T13:49:43.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.314+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-12-09T13:49:43.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.314+0000 7f90f7f39ec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T13:49:43.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.314+0000 7f90f7f39ec0 20 get_system_obj_state: rctx=0x7fffb1b67928 obj=default.rgw.meta:users.keys:NLHORHQOCYKKTUEZLGGT state=0x55a5174c3e40 s->prefetch_data=0 2019-12-09T13:49:43.318 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.314+0000 7f90f7f39ec0 10 cache get: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT : miss 2019-12-09T13:49:43.326 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.322+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT info.flags=0x0 2019-12-09T13:49:43.327 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.322+0000 7f90f7f39ec0 10 adding default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT to cache LRU end 2019-12-09T13:49:43.327 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.322+0000 7f90f7f39ec0 20 get_system_obj_state: rctx=0x7fffb1b67538 obj=default.rgw.meta:users.keys:NLHORHQOCYKKTUEZLGGT state=0x55a5174c3e40 s->prefetch_data=0 2019-12-09T13:49:43.327 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.322+0000 7f90f7f39ec0 10 cache get: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:43.345 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.332+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT info.flags=0x0 2019-12-09T13:49:43.345 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.332+0000 7f90f7f39ec0 10 moving default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT to cache LRU end 2019-12-09T13:49:43.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.332+0000 7f90f7f39ec0 20 get_system_obj_state: rctx=0x7fffb1b676b8 obj=default.rgw.meta:users.keys:NLHORHQOCYKKTUEZLGGT state=0x55a5174c3e40 s->prefetch_data=0 2019-12-09T13:49:43.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.332+0000 7f90f7f39ec0 10 cache get: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT : type miss (requested=0x6, cached=0x0) 2019-12-09T13:49:43.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.340+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT info.flags=0x0 2019-12-09T13:49:43.346 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.340+0000 7f90f7f39ec0 10 moving default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT to cache LRU end 2019-12-09T13:49:43.422 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.417+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T13:49:43.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.417+0000 7f90f7f39ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:43.423 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.417+0000 7f90f7f39ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-12-09T13:49:43.432 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.428+0000 7f90c67fc700 10 RGWWatcher::handle_notify() notify_id 592705486915 cookie 94167548244112 notifier 4993 bl.length()=691 2019-12-09T13:49:43.433 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.428+0000 7f90c67fc700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T13:49:43.433 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.428+0000 7f90c67fc700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:43.537 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.533+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:43.537 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.533+0000 7f90f7f39ec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T13:49:43.537 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.533+0000 7f90f7f39ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-12-09T13:49:43.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.546+0000 7f90c67fc700 10 RGWWatcher::handle_notify() notify_id 592705486913 cookie 94167548245104 notifier 4993 bl.length()=224 2019-12-09T13:49:43.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.546+0000 7f90c67fc700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-12-09T13:49:43.550 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.546+0000 7f90c67fc700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-12-09T13:49:43.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.744+0000 7f90f7f39ec0 10 cache put: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT info.flags=0x7 2019-12-09T13:49:43.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.744+0000 7f90f7f39ec0 10 moving default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT to cache LRU end 2019-12-09T13:49:43.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.744+0000 7f90f7f39ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=201 2019-12-09T13:49:43.772 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.767+0000 7f90c67fc700 10 RGWWatcher::handle_notify() notify_id 592705486912 cookie 94167548242256 notifier 4993 bl.length()=201 2019-12-09T13:49:43.772 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.767+0000 7f90c67fc700 10 cache put: name=default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT info.flags=0x7 2019-12-09T13:49:43.772 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:43.767+0000 7f90c67fc700 10 moving default.rgw.meta+users.keys+NLHORHQOCYKKTUEZLGGT to cache LRU end 2019-12-09T13:49:44.448 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T13:49:44.448 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "testx$tenanteduser.client.0", 2019-12-09T13:49:44.448 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [], 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [ 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T13:49:44.449 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "testx$tenanteduser.client.0", 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: "access_key": "NLHORHQOCYKKTUEZLGGT", 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "0cdUYuWXWVo1bMGbtHyEamfj2SOLLZXzxYhmbOnmWarGT6JiF5+HHg==" 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [], 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T13:49:44.450 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:44.451 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T13:49:44.452 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T13:49:44.453 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T13:49:44.454 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T13:49:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:44.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.505+0000 7f90f7f39ec0 20 remove_watcher() i=0 2019-12-09T13:49:44.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.505+0000 7f90f7f39ec0 2 removed watcher, disabling cache 2019-12-09T13:49:44.554 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.550+0000 7f90f7f39ec0 20 remove_watcher() i=1 2019-12-09T13:49:44.592 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.588+0000 7f90f7f39ec0 20 remove_watcher() i=2 2019-12-09T13:49:44.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.625+0000 7f90f7f39ec0 20 remove_watcher() i=3 2019-12-09T13:49:44.664 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.660+0000 7f90f7f39ec0 20 remove_watcher() i=4 2019-12-09T13:49:44.698 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.694+0000 7f90f7f39ec0 20 remove_watcher() i=5 2019-12-09T13:49:44.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.732+0000 7f90f7f39ec0 20 remove_watcher() i=6 2019-12-09T13:49:44.775 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.769+0000 7f90f7f39ec0 20 remove_watcher() i=7 2019-12-09T13:49:44.781 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 0230901794 --totp-seed 62UGTSWGYBPGCYR4AQQ3NZYTEEPLP6CVA4Y5GGTMU3FZYK2CNRXAUGS73IPXKQAR --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-12-09T13:49:44.906 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.901+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:44.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.940+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:44.945 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.940+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:44.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.948+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:44.952 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.948+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:44.960 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.957+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:44.961 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.957+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:44.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:44.978+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:45.163 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.159+0000 7ff1ca714ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T13:49:45.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.159+0000 7ff1ca714ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T13:49:45.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.159+0000 7ff1ca714ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T13:49:45.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.159+0000 7ff1ca714ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T13:49:45.164 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.159+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.182+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:45.186 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.182+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.196 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.192+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:45.196 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.192+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.206+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:45.210 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.206+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.214+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:45.218 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.214+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.230 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.226+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:45.231 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.226+0000 7ff1ca714ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T13:49:45.231 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.226+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.302 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.298+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:45.302 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.298+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.313 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.309+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:45.313 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.309+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.328 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.324+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T13:49:45.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.324+0000 7ff1ca714ec0 20 zonegroup default 2019-12-09T13:49:45.329 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.324+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.334 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.330+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:45.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.331+0000 7ff1ca714ec0 10 Cannot find current period zone using local zone 2019-12-09T13:49:45.335 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.331+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.336+0000 7ff1ca714ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T13:49:45.340 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.336+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.350 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.346+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T13:49:45.351 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.346+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:45.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.358+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T13:49:45.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.358+0000 7ff1ca714ec0 20 zone default found 2019-12-09T13:49:45.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.358+0000 7ff1ca714ec0 20 started sync module instance, tier type = 2019-12-09T13:49:45.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.358+0000 7ff1ca714ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T13:49:45.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=0 2019-12-09T13:49:45.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=1 2019-12-09T13:49:45.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=2 2019-12-09T13:49:45.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=3 2019-12-09T13:49:45.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=4 2019-12-09T13:49:45.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=5 2019-12-09T13:49:45.655 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.650+0000 7ff1ca714ec0 20 add_watcher() i=6 2019-12-09T13:49:45.689 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.685+0000 7ff1ca714ec0 20 add_watcher() i=7 2019-12-09T13:49:45.689 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.685+0000 7ff1ca714ec0 2 all 8 watchers are set, enabling cache 2019-12-09T13:49:45.689 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.685+0000 7ff10bfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T13:49:45.690 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.686+0000 7ff1b813f700 0 -- >> v1:172.21.15.174:6789/0 conn(0x561c580f8fc0 legacy=0x561c580fc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T13:49:45.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:45.732+0000 7ff1b813f700 0 -- 172.21.15.174:0/1021325951 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7ff0f8022470 msgr2=0x7ff0f8024960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-12-09T13:49:46.970 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.967+0000 7ff0f67fc700 20 reqs_thread_entry: start 2019-12-09T13:49:46.971 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.967+0000 7ff0effff700 20 reqs_thread_entry: start 2019-12-09T13:49:46.971 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.967+0000 7ff1ca714ec0 20 init_complete bucket index max shards: 0 2019-12-09T13:49:46.971 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.968+0000 7ff0ee7fc700 20 reqs_thread_entry: start 2019-12-09T13:49:46.972 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.968+0000 7ff1ca714ec0 20 get_system_obj_state: rctx=0x7ffd81d74488 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x561c5817ab40 s->prefetch_data=0 2019-12-09T13:49:46.972 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.968+0000 7ff1ca714ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T13:49:46.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T13:49:46.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:46.982 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T13:49:46.983 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T13:49:46.983 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T13:49:46.983 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.979+0000 7ff1ca714ec0 20 rados->read ofs=0 len=0 2019-12-09T13:49:46.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.992+0000 7ff1ca714ec0 20 rados_obj.operate() r=0 bl.length=491 2019-12-09T13:49:46.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.992+0000 7ff1ca714ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T13:49:46.995 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:46.992+0000 7ff1ca714ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:47.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.098+0000 7ff1ca714ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T13:49:47.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.098+0000 7ff1ca714ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:47.101 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.098+0000 7ff1ca714ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-12-09T13:49:47.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.107+0000 7ff1ad7fa700 10 RGWWatcher::handle_notify() notify_id 592705486916 cookie 94679736431792 notifier 4691 bl.length()=705 2019-12-09T13:49:47.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.107+0000 7ff1ad7fa700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-12-09T13:49:47.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.107+0000 7ff1ad7fa700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T13:49:47.216 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.213+0000 7ff1ca714ec0 20 remove_watcher() i=0 2019-12-09T13:49:47.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.213+0000 7ff1ca714ec0 2 removed watcher, disabling cache 2019-12-09T13:49:47.268 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.265+0000 7ff1ca714ec0 20 remove_watcher() i=1 2019-12-09T13:49:47.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.308+0000 7ff1ca714ec0 20 remove_watcher() i=2 2019-12-09T13:49:47.366 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.363+0000 7ff1ca714ec0 20 remove_watcher() i=3 2019-12-09T13:49:47.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.422+0000 7ff1ca714ec0 20 remove_watcher() i=4 2019-12-09T13:49:47.475 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.472+0000 7ff1ca714ec0 20 remove_watcher() i=5 2019-12-09T13:49:47.519 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.516+0000 7ff1ca714ec0 20 remove_watcher() i=6 2019-12-09T13:49:47.552 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T13:49:47.549+0000 7ff1ca714ec0 20 remove_watcher() i=7 2019-12-09T13:49:47.558 INFO:tasks.s3tests:Configuring s3-tests... 2019-12-09T13:49:47.558 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-12-09T13:49:53.416 INFO:teuthology.orchestra.run.smithi174.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-12-09T13:49:53.546 INFO:teuthology.orchestra.run.smithi174.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-12-09T13:49:53.587 INFO:teuthology.orchestra.run.smithi174.stdout:Loading mirror speeds from cached hostfile 2019-12-09T13:49:53.587 INFO:teuthology.orchestra.run.smithi174.stdout: * base: mirror.fileplanet.com 2019-12-09T13:49:53.587 INFO:teuthology.orchestra.run.smithi174.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-12-09T13:49:53.587 INFO:teuthology.orchestra.run.smithi174.stdout: * extras: mirrors.tripadvisor.com 2019-12-09T13:49:53.588 INFO:teuthology.orchestra.run.smithi174.stdout: * updates: mirror.cogentco.com 2019-12-09T13:49:54.341 INFO:teuthology.orchestra.run.smithi174.stdout:7 packages excluded due to repository priority protections 2019-12-09T13:49:54.705 INFO:teuthology.orchestra.run.smithi174.stdout:Resolving Dependencies 2019-12-09T13:49:54.705 INFO:teuthology.orchestra.run.smithi174.stdout:--> Running transaction check 2019-12-09T13:49:54.705 INFO:teuthology.orchestra.run.smithi174.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-12-09T13:49:55.094 INFO:teuthology.orchestra.run.smithi174.stdout:--> Finished Dependency Resolution 2019-12-09T13:49:55.331 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:55.331 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies Resolved 2019-12-09T13:49:55.332 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:55.332 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T13:49:55.332 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2019-12-09T13:49:55.332 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T13:49:55.332 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout:Install 1 Package 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:55.333 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 23 k 2019-12-09T13:49:55.334 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 27 k 2019-12-09T13:49:55.334 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading packages: 2019-12-09T13:49:55.578 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2019-12-09T13:49:55.584 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2019-12-09T13:49:55.586 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded 2019-12-09T13:49:55.587 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2019-12-09T13:49:56.181 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T13:49:56.539 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-12-09T13:49:56.539 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:56.539 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2019-12-09T13:49:56.539 INFO:teuthology.orchestra.run.smithi174.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-12-09T13:49:56.540 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:49:56.540 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2019-12-09T13:49:56.734 INFO:teuthology.orchestra.run.smithi174.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-12-09T13:49:58.204 INFO:teuthology.orchestra.run.smithi174.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T13:49:58.532 INFO:teuthology.orchestra.run.smithi174.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T13:49:58.652 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting pip 2019-12-09T13:49:58.730 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T13:49:58.793 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: pip 2019-12-09T13:49:58.793 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: pip 9.0.1 2019-12-09T13:49:58.852 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling pip-9.0.1: 2019-12-09T13:49:58.892 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T13:49:59.471 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed pip-19.3.1 2019-12-09T13:49:59.771 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:50:00.221 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting setuptools==32.3.1 2019-12-09T13:50:00.224 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T13:50:00.264 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: setuptools 2019-12-09T13:50:00.264 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T13:50:00.281 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T13:50:00.290 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T13:50:00.417 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed setuptools-32.3.1 2019-12-09T13:50:00.895 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T13:50:01.076 INFO:teuthology.orchestra.run.smithi174.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T13:50:01.139 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting nose>=1.0.0 2019-12-09T13:50:01.143 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T13:50:01.313 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting boto>=2.6.0 2019-12-09T13:50:01.321 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T13:50:01.819 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting boto3>=1.0.0 2019-12-09T13:50:01.883 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/1f/a5/6b25e39aea40bc03163615c07572d90aaddc9951f5af9c21204d3da46398/boto3-1.10.34-py2.py3-none-any.whl (128kB) 2019-12-09T13:50:01.991 INFO:teuthology.orchestra.run.smithi174.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T13:50:02.262 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting gevent>=1.0 2019-12-09T13:50:02.276 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:50:02.538 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting isodate>=0.4.4 2019-12-09T13:50:02.547 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-12-09T13:50:02.685 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting requests>=0.14.0 2019-12-09T13:50:02.695 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-12-09T13:50:02.967 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting pytz>=2011k 2019-12-09T13:50:02.977 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-12-09T13:50:03.181 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting ordereddict 2019-12-09T13:50:03.191 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-12-09T13:50:03.460 INFO:teuthology.orchestra.run.smithi174.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-12-09T13:50:03.853 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting lxml 2019-12-09T13:50:03.870 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/25/73/730ad7249847f741c7e622f52b971daa5540f6fb87589bc92c717f5aafba/lxml-4.4.2-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:50:04.076 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-12-09T13:50:04.087 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-12-09T13:50:04.937 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting botocore<1.14.0,>=1.13.34 2019-12-09T13:50:04.948 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/59/cb/b4772a4abc128f6fd637af9d7fc93a1db11617859af680fc2b2e6282eb95/botocore-1.13.34-py2.py3-none-any.whl (5.8MB) 2019-12-09T13:50:05.473 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-12-09T13:50:05.484 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-12-09T13:50:05.652 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T13:50:05.660 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T13:50:05.841 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting six 2019-12-09T13:50:05.853 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T13:50:05.969 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-12-09T13:50:05.979 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-12-09T13:50:06.084 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting idna<2.9,>=2.5 2019-12-09T13:50:06.094 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-12-09T13:50:06.268 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-12-09T13:50:06.277 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-12-09T13:50:06.435 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting certifi>=2017.4.17 2019-12-09T13:50:06.446 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/b9/63/df50cac98ea0d5b006c55a399c3bf1db9da7b5a24de7890bc9cfd5dd9e99/certifi-2019.11.28-py2.py3-none-any.whl (156kB) 2019-12-09T13:50:06.556 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting docutils<0.16,>=0.10 2019-12-09T13:50:06.566 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-12-09T13:50:06.734 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-12-09T13:50:06.743 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-12-09T13:50:06.839 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-12-09T13:50:06.848 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-12-09T13:50:06.880 INFO:teuthology.orchestra.run.smithi174.stdout:Building wheels for collected packages: ordereddict 2019-12-09T13:50:06.880 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for ordereddict (setup.py): started 2019-12-09T13:50:07.105 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-12-09T13:50:07.105 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=346534c2a577eb1780c24edafe03e9e60f7e2e24b6e1c1a554f9c69a95ed5d57 2019-12-09T13:50:07.105 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-12-09T13:50:07.108 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully built ordereddict 2019-12-09T13:50:07.188 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-12-09T13:50:09.329 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed PyYAML-5.2 boto-2.49.0 boto3-1.10.34 botocore-1.13.34 bunch-1.0.1 certifi-2019.11.28 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.2 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-12-09T13:50:09.625 INFO:teuthology.orchestra.run.smithi174.stdout:running develop 2019-12-09T13:50:09.677 INFO:teuthology.orchestra.run.smithi174.stdout:running egg_info 2019-12-09T13:50:09.677 INFO:teuthology.orchestra.run.smithi174.stdout:creating s3tests.egg-info 2019-12-09T13:50:09.677 INFO:teuthology.orchestra.run.smithi174.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-12-09T13:50:09.678 INFO:teuthology.orchestra.run.smithi174.stdout:writing s3tests.egg-info/PKG-INFO 2019-12-09T13:50:09.678 INFO:teuthology.orchestra.run.smithi174.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-12-09T13:50:09.678 INFO:teuthology.orchestra.run.smithi174.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-12-09T13:50:09.678 INFO:teuthology.orchestra.run.smithi174.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-12-09T13:50:09.679 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T13:50:09.680 INFO:teuthology.orchestra.run.smithi174.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T13:50:09.681 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-12-09T13:50:09.682 INFO:teuthology.orchestra.run.smithi174.stdout:running build_ext 2019-12-09T13:50:09.682 INFO:teuthology.orchestra.run.smithi174.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-12-09T13:50:09.682 INFO:teuthology.orchestra.run.smithi174.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-12-09T13:50:09.685 INFO:teuthology.orchestra.run.smithi174.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T13:50:09.685 INFO:teuthology.orchestra.run.smithi174.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T13:50:09.685 INFO:teuthology.orchestra.run.smithi174.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T13:50:09.685 INFO:teuthology.orchestra.run.smithi174.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T13:50:09.686 INFO:teuthology.orchestra.run.smithi174.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-12-09T13:50:09.686 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.686 INFO:teuthology.orchestra.run.smithi174.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-12-09T13:50:09.686 INFO:teuthology.orchestra.run.smithi174.stdout:Processing dependencies for s3tests==0.0.1 2019-12-09T13:50:09.733 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for isodate==0.6.0 2019-12-09T13:50:09.733 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: isodate 0.6.0 2019-12-09T13:50:09.734 INFO:teuthology.orchestra.run.smithi174.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-12-09T13:50:09.734 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.734 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.736 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for gevent==1.4.0 2019-12-09T13:50:09.736 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: gevent 1.4.0 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for bunch==1.0.1 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: bunch 1.0.1 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-12-09T13:50:09.737 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.739 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.740 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for PyYAML==5.2 2019-12-09T13:50:09.740 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: PyYAML 5.2 2019-12-09T13:50:09.740 INFO:teuthology.orchestra.run.smithi174.stdout:Adding PyYAML 5.2 to easy-install.pth file 2019-12-09T13:50:09.740 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.740 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.741 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for boto3==1.10.34 2019-12-09T13:50:09.741 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: boto3 1.10.34 2019-12-09T13:50:09.742 INFO:teuthology.orchestra.run.smithi174.stdout:Adding boto3 1.10.34 to easy-install.pth file 2019-12-09T13:50:09.742 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.742 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.742 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for boto==2.49.0 2019-12-09T13:50:09.742 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: boto 2.49.0 2019-12-09T13:50:09.745 INFO:teuthology.orchestra.run.smithi174.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-12-09T13:50:09.745 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.745 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.745 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for six==1.13.0 2019-12-09T13:50:09.745 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: six 1.13.0 2019-12-09T13:50:09.746 INFO:teuthology.orchestra.run.smithi174.stdout:Adding six 1.13.0 to easy-install.pth file 2019-12-09T13:50:09.746 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.746 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.747 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for greenlet==0.4.15 2019-12-09T13:50:09.747 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: greenlet 0.4.15 2019-12-09T13:50:09.747 INFO:teuthology.orchestra.run.smithi174.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-12-09T13:50:09.747 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.747 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.748 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for s3transfer==0.2.1 2019-12-09T13:50:09.748 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: s3transfer 0.2.1 2019-12-09T13:50:09.748 INFO:teuthology.orchestra.run.smithi174.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-12-09T13:50:09.748 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.749 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.749 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for botocore==1.13.34 2019-12-09T13:50:09.749 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: botocore 1.13.34 2019-12-09T13:50:09.749 INFO:teuthology.orchestra.run.smithi174.stdout:Adding botocore 1.13.34 to easy-install.pth file 2019-12-09T13:50:09.749 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.750 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.750 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for jmespath==0.9.4 2019-12-09T13:50:09.750 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: jmespath 0.9.4 2019-12-09T13:50:09.750 INFO:teuthology.orchestra.run.smithi174.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-12-09T13:50:09.751 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.751 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.751 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for futures==3.3.0 2019-12-09T13:50:09.751 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: futures 3.3.0 2019-12-09T13:50:09.751 INFO:teuthology.orchestra.run.smithi174.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-12-09T13:50:09.752 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.752 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.754 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for urllib3==1.25.7 2019-12-09T13:50:09.755 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: urllib3 1.25.7 2019-12-09T13:50:09.755 INFO:teuthology.orchestra.run.smithi174.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-12-09T13:50:09.755 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.755 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.755 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for python-dateutil==2.8.0 2019-12-09T13:50:09.756 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: python-dateutil 2.8.0 2019-12-09T13:50:09.756 INFO:teuthology.orchestra.run.smithi174.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-12-09T13:50:09.756 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.756 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.756 INFO:teuthology.orchestra.run.smithi174.stdout:Searching for docutils==0.15.2 2019-12-09T13:50:09.757 INFO:teuthology.orchestra.run.smithi174.stdout:Best match: docutils 0.15.2 2019-12-09T13:50:09.757 INFO:teuthology.orchestra.run.smithi174.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-12-09T13:50:09.757 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T13:50:09.757 INFO:teuthology.orchestra.run.smithi174.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-12-09T13:50:09.757 INFO:teuthology.orchestra.run.smithi174.stdout:Finished processing dependencies for s3tests==0.0.1 2019-12-09T13:50:09.759 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-12-09T13:50:09.828 INFO:tasks.s3tests:Configuring boto... 2019-12-09T13:50:09.830 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/boto.cfg 2019-12-09T13:50:09.937 INFO:teuthology.orchestra.run.smithi174:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-12-09T13:50:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-12-09T13:50:20.202 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-12-09T13:50:24.417 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-12-09T13:50:27.863 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-12-09T13:50:30.623 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-12-09T13:50:33.134 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-12-09T13:50:37.903 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-12-09T13:50:40.023 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-12-09T13:50:40.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-12-09T13:50:40.977 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-12-09T13:50:44.479 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-12-09T13:50:47.460 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-12-09T13:50:49.788 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T13:50:51.211 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-12-09T13:50:51.456 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-12-09T13:50:51.665 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-12-09T13:50:51.665 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-12-09T13:50:51.667 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-12-09T13:50:51.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-12-09T13:50:51.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T13:50:51.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T13:50:51.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-12-09T13:50:51.670 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T13:50:51.670 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T13:50:51.670 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T13:50:51.680 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-12-09T13:50:51.681 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-12-09T13:50:51.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-12-09T13:50:51.683 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-12-09T13:50:51.683 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-12-09T13:50:51.683 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-12-09T13:50:51.683 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-12-09T13:50:51.683 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-12-09T13:50:51.693 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-12-09T13:50:51.693 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-12-09T13:50:51.693 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-12-09T13:50:51.693 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-12-09T13:51:00.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-12-09T13:51:00.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-12-09T13:51:00.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-12-09T13:51:00.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-12-09T13:51:00.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-12-09T13:51:11.359 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-12-09T13:51:19.410 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-12-09T13:51:31.150 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-12-09T13:51:46.300 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-12-09T13:52:00.720 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-12-09T13:52:06.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-12-09T13:52:10.691 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-12-09T13:52:15.036 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-12-09T13:52:15.036 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-12-09T13:52:15.036 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-12-09T13:52:15.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T13:52:15.038 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-12-09T13:52:15.038 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T13:52:15.041 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T13:52:15.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-12-09T13:52:15.043 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-12-09T13:52:15.044 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-12-09T13:52:15.044 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-12-09T13:52:15.044 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-12-09T13:52:15.044 INFO:teuthology.orchestra.run.smithi174.stderr:SKIP 2019-12-09T13:52:15.047 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-12-09T13:52:33.484 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:25.335 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.488 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:25.338 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.551 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:25.374 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.571 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:25.395 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.643 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:25.467 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.668 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:25.492 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.683 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:25.534 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.700 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:25.524 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:33.813 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:25.637 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.770 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:32.621 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.783 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:32.633 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.785 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:32.610 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.789 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:32.613 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.792 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:32.616 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.799 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:32.623 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.815 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:31:32.639 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.826 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:31:32.677 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:52:40.836 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:31:32.660 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:53:28.947 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T13:53:28.948 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T13:53:29.324 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T13:53:29.492 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T13:53:29.846 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T13:53:29.871 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T13:53:29.890 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T13:53:29.890 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T13:53:29.890 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T13:53:29.890 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T13:53:29.891 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T13:53:29.891 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T13:53:29.891 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T13:53:29.891 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T13:53:29.891 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T13:53:29.892 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T13:53:29.892 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T13:53:29.895 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T13:53:29.895 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T13:53:29.896 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T13:53:29.896 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T13:53:29.900 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T13:53:29.900 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T13:53:29.900 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T13:53:29.901 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T13:53:29.925 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T13:53:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T13:53:29.950 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T13:53:30.102 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T13:53:35.377 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok 2019-12-09T13:53:52.326 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... ok 2019-12-09T13:53:54.525 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... ok 2019-12-09T13:53:58.946 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... ok 2019-12-09T13:54:02.080 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... ok 2019-12-09T13:54:05.516 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... ok 2019-12-09T13:54:10.812 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... ok 2019-12-09T13:54:13.998 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... ok 2019-12-09T13:54:19.037 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok 2019-12-09T13:54:21.344 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... ok 2019-12-09T13:54:24.604 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... ok 2019-12-09T13:54:28.211 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... ok 2019-12-09T13:54:29.726 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... ok 2019-12-09T13:54:31.731 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... ok 2019-12-09T13:54:32.223 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... ok 2019-12-09T13:54:34.448 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... ok 2019-12-09T13:54:37.872 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... ok 2019-12-09T13:54:42.395 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... ok 2019-12-09T13:54:44.390 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... ok 2019-12-09T13:54:47.919 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... ok 2019-12-09T13:54:49.961 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... ok 2019-12-09T13:54:52.389 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T13:54:54.717 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... ok 2019-12-09T13:54:56.815 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... ok 2019-12-09T13:54:58.833 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... ok 2019-12-09T13:54:59.211 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... ok 2019-12-09T13:55:00.316 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... ok 2019-12-09T13:55:00.892 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... ok 2019-12-09T13:55:02.126 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... ok 2019-12-09T13:55:02.304 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... ok 2019-12-09T13:55:04.746 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... ok 2019-12-09T13:55:08.848 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... ok 2019-12-09T13:55:16.135 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... ok 2019-12-09T13:55:22.024 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... ok 2019-12-09T13:55:28.577 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... ok 2019-12-09T13:55:36.994 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... ok 2019-12-09T13:55:44.826 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... ok 2019-12-09T13:55:57.847 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... ok 2019-12-09T13:56:12.752 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... ok 2019-12-09T13:56:16.289 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T13:56:19.284 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... ok 2019-12-09T13:56:26.479 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... ok 2019-12-09T13:56:32.595 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... ok 2019-12-09T13:56:46.807 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... ok 2019-12-09T13:57:01.146 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... ok 2019-12-09T13:57:09.346 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... ok 2019-12-09T13:57:16.343 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... ok 2019-12-09T13:57:23.872 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... ok 2019-12-09T13:57:32.131 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... ok 2019-12-09T13:57:40.503 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... ok 2019-12-09T13:57:46.963 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... ok 2019-12-09T13:57:52.425 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... ok 2019-12-09T13:57:54.707 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:36:46.558 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.712 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:36:46.563 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.726 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:36:46.550 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.727 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:36:46.551 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.728 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:36:46.552 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.731 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:36:46.555 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.930 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:36:46.781 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.963 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:36:46.786 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:54.965 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:36:46.789 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T13:57:59.201 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... ok 2019-12-09T13:58:06.116 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... ok 2019-12-09T13:58:14.515 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... ok 2019-12-09T13:58:19.967 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... ok 2019-12-09T13:58:25.766 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... ok 2019-12-09T13:58:31.725 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... ok 2019-12-09T13:58:37.047 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... ok 2019-12-09T13:58:45.829 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... ok 2019-12-09T13:58:51.940 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... ok 2019-12-09T13:58:58.920 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... ok 2019-12-09T14:00:55.285 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:39:47.136 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.292 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:39:47.143 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.310 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:39:47.134 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.311 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:39:47.134 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.359 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:39:47.182 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.375 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:39:47.199 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.394 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:39:47.245 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.459 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:39:47.283 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:00:55.475 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:39:47.299 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.209 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:41:55.060 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.218 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:41:55.069 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.226 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:41:55.049 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.226 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:41:55.050 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.228 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:41:55.052 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.229 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:41:55.053 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.320 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:41:55.144 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.330 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:41:55.149 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:03:03.348 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:41:55.199 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.364 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:47:07.215 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.377 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:47:07.228 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.380 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:47:07.204 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.383 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:47:07.207 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.383 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:47:07.207 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.389 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:47:07.213 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.579 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:47:07.403 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.599 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:47:07.450 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:08:15.605 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:47:07.429 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.231 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:48:09.081 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.244 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:48:09.095 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.268 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:48:09.092 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.280 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:48:09.104 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.289 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:48:09.113 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.292 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:48:09.143 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.304 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:48:09.128 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.333 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:48:09.157 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:09:17.343 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:48:09.167 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.568 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:52:19.419 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.575 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:52:19.426 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.583 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:52:19.407 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.584 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:52:19.408 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.586 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:52:19.410 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.593 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:52:19.414 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.732 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:52:19.556 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.733 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:52:19.583 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:13:27.784 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:52:19.608 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.175 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:56:31.025 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.199 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:56:31.034 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.257 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:56:31.081 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.309 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:56:31.083 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.309 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:56:31.087 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.309 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:56:31.094 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.323 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:56:31.174 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.362 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:56:31.186 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:17:39.397 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:56:31.221 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.934 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:57:30.785 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.940 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:57:30.791 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.955 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:57:30.779 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.955 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:57:30.779 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.960 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:57:30.784 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:38.964 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:57:30.788 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:39.009 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:02:57:30.860 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:39.017 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:02:57:30.842 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:18:39.033 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:02:57:30.852 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:20:53.789 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... ok 2019-12-09T14:21:00.757 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... ok 2019-12-09T14:21:08.900 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... ok 2019-12-09T14:21:15.549 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... ok 2019-12-09T14:21:23.647 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... ok 2019-12-09T14:21:30.724 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... ok 2019-12-09T14:21:38.652 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... ok 2019-12-09T14:21:45.946 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... ok 2019-12-09T14:21:51.689 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... ok 2019-12-09T14:21:58.548 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... ok 2019-12-09T14:22:05.196 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... ok 2019-12-09T14:22:11.816 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... ok 2019-12-09T14:22:18.701 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... ok 2019-12-09T14:22:28.965 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... ok 2019-12-09T14:22:37.187 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... ok 2019-12-09T14:22:47.394 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... ok 2019-12-09T14:22:55.957 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... ok 2019-12-09T14:23:05.530 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T14:23:11.362 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... ok 2019-12-09T14:23:17.728 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T14:23:23.138 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... ok 2019-12-09T14:23:30.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T14:23:37.798 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-12-09T14:23:44.272 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... ok 2019-12-09T14:23:50.119 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:02:41.970 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.129 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:02:41.980 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.137 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:02:41.961 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.138 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:02:41.962 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.142 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:02:41.966 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.144 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:02:41.967 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.318 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:02:42.142 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.341 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:02:42.192 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.347 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:02:42.171 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:23:50.814 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... ok 2019-12-09T14:23:56.885 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... ok 2019-12-09T14:24:03.493 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... ok 2019-12-09T14:24:10.884 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:03:02.733 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.895 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:03:02.719 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.897 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:03:02.721 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.905 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:03:02.754 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.939 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:03:02.763 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.941 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:03:02.792 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.947 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:03:02.771 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.974 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:03:02.798 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:10.994 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:03:02.818 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:24:11.193 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... ok 2019-12-09T14:24:16.109 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... ok 2019-12-09T14:24:52.023 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... ok 2019-12-09T14:25:32.103 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... ok 2019-12-09T14:25:40.298 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... ok 2019-12-09T14:25:49.383 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... ok 2019-12-09T14:25:58.078 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... ok 2019-12-09T14:26:01.423 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:04:53.274 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.428 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:04:53.279 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.455 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:04:53.279 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.461 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:04:53.285 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.470 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:04:53.294 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.484 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:04:53.308 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.518 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:04:53.369 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.535 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:04:53.359 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:01.539 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:04:53.363 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:26:06.370 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... ok 2019-12-09T14:26:16.112 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... ok 2019-12-09T14:26:23.468 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... ok 2019-12-09T14:26:32.863 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... ok 2019-12-09T14:26:40.938 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... ok 2019-12-09T14:26:50.329 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... ok 2019-12-09T14:27:00.052 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... ok 2019-12-09T14:27:09.516 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... ok 2019-12-09T14:27:30.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... ok 2019-12-09T14:27:35.513 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... ok 2019-12-09T14:27:39.626 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... ok 2019-12-09T14:27:42.174 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... ok 2019-12-09T14:27:44.111 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... ok 2019-12-09T14:27:45.111 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... ok 2019-12-09T14:27:45.843 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... ok 2019-12-09T14:27:46.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... ok 2019-12-09T14:27:52.201 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... ok 2019-12-09T14:28:09.499 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... ok 2019-12-09T14:28:10.898 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... ok 2019-12-09T14:28:17.449 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... ok 2019-12-09T14:28:21.678 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... ok 2019-12-09T14:28:25.094 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... ok 2019-12-09T14:28:40.220 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... ok 2019-12-09T14:28:54.827 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... ok 2019-12-09T14:28:59.220 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... ok 2019-12-09T14:29:00.349 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:07:52.200 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.357 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:07:52.181 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.360 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:07:52.211 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.362 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:07:52.186 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.362 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:07:52.186 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.369 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:07:52.193 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.499 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:07:52.323 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.518 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:07:52.369 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:00.533 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:07:52.357 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:29:01.945 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... ok 2019-12-09T14:29:05.106 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... ok 2019-12-09T14:29:10.098 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... ok 2019-12-09T14:29:17.128 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... ok 2019-12-09T14:29:22.250 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... ok 2019-12-09T14:29:28.297 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... ok 2019-12-09T14:29:34.853 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... ok 2019-12-09T14:29:39.324 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... ok 2019-12-09T14:29:45.420 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... ok 2019-12-09T14:29:49.070 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... ok 2019-12-09T14:29:52.234 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... ok 2019-12-09T14:29:56.317 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... ok 2019-12-09T14:30:02.426 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... ok 2019-12-09T14:30:06.175 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... ok 2019-12-09T14:30:09.464 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... ok 2019-12-09T14:30:15.033 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... ok 2019-12-09T14:30:23.200 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... ok 2019-12-09T14:30:28.435 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... ok 2019-12-09T14:30:32.124 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... ok 2019-12-09T14:30:35.715 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... ok 2019-12-09T14:30:39.295 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... ok 2019-12-09T14:30:45.222 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... ok 2019-12-09T14:30:49.508 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... ok 2019-12-09T14:30:52.166 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... ok 2019-12-09T14:30:55.255 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... ok 2019-12-09T14:30:58.652 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... ok 2019-12-09T14:31:03.250 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... ok 2019-12-09T14:31:06.182 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... ok 2019-12-09T14:31:10.911 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... ok 2019-12-09T14:31:14.287 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... ok 2019-12-09T14:31:18.737 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... ok 2019-12-09T14:31:21.509 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... ok 2019-12-09T14:31:24.451 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... ok 2019-12-09T14:31:28.166 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... ok 2019-12-09T14:31:30.080 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... ok 2019-12-09T14:31:33.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... ok 2019-12-09T14:31:36.386 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... ok 2019-12-09T14:31:39.122 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... ok 2019-12-09T14:31:41.811 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... ok 2019-12-09T14:31:44.886 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... ok 2019-12-09T14:31:48.675 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... ok 2019-12-09T14:31:54.050 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... ok 2019-12-09T14:31:56.782 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... ok 2019-12-09T14:32:00.765 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... ok 2019-12-09T14:32:05.689 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... ok 2019-12-09T14:32:09.904 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... ok 2019-12-09T14:32:14.734 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... ok 2019-12-09T14:32:20.289 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... ok 2019-12-09T14:32:24.907 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... ok 2019-12-09T14:32:30.575 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... ok 2019-12-09T14:32:39.597 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... ok 2019-12-09T14:32:45.802 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok 2019-12-09T14:32:50.419 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... ok 2019-12-09T14:32:59.132 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... ok 2019-12-09T14:33:05.232 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... ok 2019-12-09T14:33:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok 2019-12-09T14:33:15.075 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... ok 2019-12-09T14:33:17.525 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... ok 2019-12-09T14:33:29.058 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... ok 2019-12-09T14:33:38.933 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... ok 2019-12-09T14:33:44.366 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... ok 2019-12-09T14:33:46.957 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... ok 2019-12-09T14:33:56.178 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... ok 2019-12-09T14:34:01.124 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... ok 2019-12-09T14:34:05.047 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... ok 2019-12-09T14:34:10.022 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... ok 2019-12-09T14:34:13.168 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:05.019 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.172 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:05.023 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.184 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:05.008 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.188 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:05.012 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.261 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:05.085 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.265 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:05.090 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.361 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:05.185 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.381 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:05.232 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.445 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:05.269 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:13.980 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... ok 2019-12-09T14:34:17.286 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... ok 2019-12-09T14:34:20.325 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... ok 2019-12-09T14:34:23.213 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:15.065 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.216 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:15.067 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.237 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:13:15.088 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.238 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:15.061 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.250 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:15.075 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.269 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:15.093 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.277 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:15.102 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.291 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:13:15.116 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:23.303 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:13:15.127 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:34:28.374 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... ok 2019-12-09T14:34:32.698 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... ok 2019-12-09T14:34:36.158 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... ok 2019-12-09T14:34:39.364 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... ok 2019-12-09T14:34:43.623 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... ok 2019-12-09T14:34:47.486 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... ok 2019-12-09T14:34:53.336 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... ok 2019-12-09T14:34:59.448 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... ok 2019-12-09T14:35:03.794 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... ok 2019-12-09T14:35:07.495 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... ok 2019-12-09T14:35:08.482 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... ok 2019-12-09T14:35:09.196 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... ok 2019-12-09T14:35:09.789 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... ok 2019-12-09T14:35:10.230 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... ok 2019-12-09T14:35:16.841 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... ok 2019-12-09T14:35:20.701 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... ok 2019-12-09T14:35:26.167 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... ok 2019-12-09T14:35:29.215 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... ok 2019-12-09T14:35:33.016 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... ok 2019-12-09T14:35:38.258 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... ok 2019-12-09T14:35:40.675 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... ok 2019-12-09T14:35:43.739 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... ok 2019-12-09T14:35:48.636 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... ok 2019-12-09T14:35:54.044 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... ok 2019-12-09T14:35:59.100 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... ok 2019-12-09T14:36:04.091 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... ok 2019-12-09T14:36:09.092 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... ok 2019-12-09T14:36:15.286 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... ok 2019-12-09T14:36:24.965 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... ok 2019-12-09T14:36:31.368 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... ok 2019-12-09T14:36:40.214 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... ok 2019-12-09T14:36:44.359 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... ok 2019-12-09T14:36:50.668 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... ok 2019-12-09T14:36:58.210 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... ok 2019-12-09T14:37:03.975 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... ok 2019-12-09T14:37:10.415 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... ok 2019-12-09T14:37:17.867 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... ok 2019-12-09T14:37:28.649 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... ok 2019-12-09T14:37:35.275 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... ok 2019-12-09T14:37:41.143 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... ok 2019-12-09T14:37:47.918 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... ok 2019-12-09T14:37:55.042 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... ok 2019-12-09T14:37:58.133 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... ok 2019-12-09T14:38:06.010 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... ok 2019-12-09T14:38:10.415 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... ok 2019-12-09T14:38:16.640 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... ok 2019-12-09T14:38:20.476 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... ok 2019-12-09T14:38:23.996 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... ok 2019-12-09T14:38:30.583 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... ok 2019-12-09T14:38:40.682 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... ok 2019-12-09T14:38:50.980 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... ok 2019-12-09T14:39:00.654 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... ok 2019-12-09T14:39:09.720 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... ok 2019-12-09T14:39:20.399 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... ok 2019-12-09T14:39:25.429 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:18:17.280 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.440 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:18:17.291 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.447 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:18:17.271 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.449 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:18:17.273 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.451 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:18:17.276 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.452 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:18:17.276 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.619 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:18:17.470 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.623 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:18:17.447 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:25.649 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:18:17.473 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:39:31.261 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... ok 2019-12-09T14:39:42.490 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... ok 2019-12-09T14:39:54.387 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... ok 2019-12-09T14:40:05.901 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... ok 2019-12-09T14:40:19.659 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... ok 2019-12-09T14:40:33.446 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... ok 2019-12-09T14:40:50.002 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... ok 2019-12-09T14:41:06.563 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... ok 2019-12-09T14:41:07.519 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... ok 2019-12-09T14:41:08.967 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... ok 2019-12-09T14:41:09.446 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... ok 2019-12-09T14:41:20.603 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... ok 2019-12-09T14:41:33.535 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok 2019-12-09T14:41:37.571 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok 2019-12-09T14:41:40.389 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok 2019-12-09T14:41:50.757 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... ok 2019-12-09T14:42:35.261 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... ok 2019-12-09T14:42:45.190 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:21:37.040 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.197 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:21:37.048 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.256 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:21:37.080 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.265 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:21:37.089 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.284 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:21:37.108 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.291 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:21:37.115 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.311 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:21:37.162 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.407 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:21:37.231 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:45.409 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:21:37.233 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:42:48.703 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... ok 2019-12-09T14:42:56.467 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... ok 2019-12-09T14:43:04.690 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... ok 2019-12-09T14:43:11.380 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... ok 2019-12-09T14:43:15.318 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... ok 2019-12-09T14:43:20.992 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok 2019-12-09T14:43:29.080 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... ok 2019-12-09T14:43:35.422 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... ok 2019-12-09T14:43:43.450 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... ok 2019-12-09T14:43:51.308 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... ok 2019-12-09T14:44:05.702 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... ok 2019-12-09T14:44:05.714 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:44:05.702+0000 ac34f700 -1 handle_data failed with exit code -104 2019-12-09T14:44:18.727 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... ok 2019-12-09T14:44:18.766 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:44:18.744+0000 86303700 -1 handle_data failed with exit code -32 2019-12-09T14:44:22.233 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... ok 2019-12-09T14:44:25.614 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... ok 2019-12-09T14:44:36.826 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:23:28.677 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.870 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:23:28.721 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.920 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:23:28.744 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.922 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:23:28.746 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.923 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:23:28.747 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.924 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:23:28.748 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:36.958 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:23:28.808 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:37.016 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:23:28.840 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:44:37.022 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:23:28.846 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:45:00.155 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... ok 2019-12-09T14:45:14.557 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... ok 2019-12-09T14:46:07.298 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:46:07.225+0000 43669700 -1 handle_data failed with exit code -104 2019-12-09T14:47:25.115 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... ok 2019-12-09T14:47:28.306 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... ok 2019-12-09T14:47:40.846 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... ok 2019-12-09T14:48:03.371 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:48:03.282+0000 44e6c700 -1 handle_data failed with exit code -104 2019-12-09T14:48:04.382 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... ok 2019-12-09T14:48:11.530 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... ok 2019-12-09T14:48:28.793 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... ok 2019-12-09T14:48:50.210 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:48:50.055+0000 88307700 -1 handle_data failed with exit code -104 2019-12-09T14:49:07.795 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:49:07.750+0000 be373700 -1 handle_data failed with exit code -104 2019-12-09T14:49:24.667 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:49:24.641+0000 682c7700 -1 handle_data failed with exit code -104 2019-12-09T14:49:45.789 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... ok 2019-12-09T14:49:48.992 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:28:40.843 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.020 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:28:40.871 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.022 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:28:40.846 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.033 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:28:40.857 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.047 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:28:40.871 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.053 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:28:40.877 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.217 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:28:41.041 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.245 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:28:41.069 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:49:49.269 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:28:41.120 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:50:52.273 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:50:52.226+0000 125441700 -1 handle_data failed with exit code -104 2019-12-09T14:51:07.280 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:29:59.130 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.283 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:29:59.134 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.286 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:29:59.110 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.290 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:29:59.114 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.336 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:29:59.160 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.338 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:29:59.163 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.355 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:29:59.179 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.357 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:29:59.208 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:07.386 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:29:59.210 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:51:42.540 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:51:42.517+0000 11c42f700 -1 handle_data failed with exit code -104 2019-12-09T14:51:55.577 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... ok 2019-12-09T14:53:17.110 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:53:17.039+0000 86303700 -1 handle_data failed with exit code -104 2019-12-09T14:53:41.186 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:53:41.144+0000 125c42700 -1 handle_data failed with exit code -104 2019-12-09T14:53:43.227 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:53:43.193+0000 bd371700 -1 handle_data failed with exit code -104 2019-12-09T14:54:08.310 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:54:08.279+0000 97b26700 -1 handle_data failed with exit code -104 2019-12-09T14:54:11.212 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:54:11.192+0000 ddbb2700 -1 handle_data failed with exit code -104 2019-12-09T14:54:20.364 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... ok 2019-12-09T14:55:15.976 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:55:15.960+0000 fdbf2700 -1 handle_data failed with exit code -104 2019-12-09T14:55:55.395 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:55:55.376+0000 b2b5c700 -1 handle_data failed with exit code -104 2019-12-09T14:57:02.729 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:57:02.686+0000 4be7a700 -1 handle_data failed with exit code -104 2019-12-09T14:57:47.579 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:57:47.560+0000 ab34d700 -1 handle_data failed with exit code -104 2019-12-09T14:58:57.480 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:58:57.453+0000 5668f700 -1 handle_data failed with exit code -104 2019-12-09T14:59:29.405 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:38:21.256 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.431 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:38:21.256 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.436 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:38:21.261 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.458 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:38:21.308 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.475 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:38:21.299 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.478 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:38:21.303 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.676 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:38:21.500 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.682 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:38:21.506 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:29.749 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:38:21.599 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T14:59:39.166 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T14:59:39.156+0000 4d67d700 -1 handle_data failed with exit code -104 2019-12-09T15:00:10.020 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:39:01.870 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.029 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:39:01.880 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.037 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:39:01.861 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.045 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:39:01.869 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.048 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:39:01.873 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.050 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:39:01.875 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.088 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:39:01.939 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.096 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:39:01.921 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:00:10.128 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:39:01.952 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:03:41.434 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:03:41.384+0000 8b30d700 -1 handle_data failed with exit code -104 2019-12-09T15:04:20.228 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:04:20.167+0000 128c48700 -1 handle_data failed with exit code -104 2019-12-09T15:04:29.760 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... ok 2019-12-09T15:05:21.311 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:44:13.161 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.320 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:44:13.171 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.334 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:44:13.158 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.335 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:44:13.160 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.337 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:44:13.162 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.340 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:44:13.165 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.531 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:44:13.356 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.545 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:44:13.370 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:05:21.574 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:44:13.425 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:06:12.332 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... ok 2019-12-09T15:06:39.887 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:06:39.834+0000 eabcc700 -1 handle_data failed with exit code -104 2019-12-09T15:07:00.437 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:07:00.366+0000 87305700 -1 handle_data failed with exit code -104 2019-12-09T15:07:17.821 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:07:17.795+0000 7b2ed700 -1 handle_data failed with exit code -104 2019-12-09T15:07:40.131 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:07:40.086+0000 93b1e700 -1 handle_data failed with exit code -104 2019-12-09T15:07:56.729 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:46:48.580 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.760 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:46:48.584 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.771 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:46:48.622 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.781 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:46:48.606 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.786 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:46:48.610 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.796 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:46:48.620 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.844 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:46:48.694 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.855 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:46:48.680 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:07:56.865 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:46:48.690 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:08:06.825 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:08:06.790+0000 6e2d3700 -1 handle_data failed with exit code -104 2019-12-09T15:08:28.307 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T15:08:28.271+0000 8e313700 -1 handle_data failed with exit code -104 2019-12-09T15:08:33.971 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... ok 2019-12-09T15:09:06.364 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... ok 2019-12-09T15:09:48.962 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... ok 2019-12-09T15:10:16.079 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok 2019-12-09T15:10:32.427 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:49:24.277 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.439 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:49:24.290 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.441 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:49:24.265 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.446 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:49:24.270 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.448 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:49:24.272 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.453 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:49:24.278 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.554 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:49:24.379 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.573 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:49:24.397 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:32.599 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:49:24.450 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:10:34.181 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... ok 2019-12-09T15:10:37.577 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... ok 2019-12-09T15:11:09.584 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... ok 2019-12-09T15:11:18.446 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... ok 2019-12-09T15:11:25.848 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... ok 2019-12-09T15:11:31.321 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... ok 2019-12-09T15:11:38.022 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... ok 2019-12-09T15:12:11.144 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... ok 2019-12-09T15:12:20.354 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... ok 2019-12-09T15:12:29.796 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... ok 2019-12-09T15:12:40.698 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... ok 2019-12-09T15:12:57.070 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... ok 2019-12-09T15:13:19.805 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... ok 2019-12-09T15:13:33.325 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... ok 2019-12-09T15:13:54.620 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... ok 2019-12-09T15:14:21.216 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... ok 2019-12-09T15:14:36.749 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... ok 2019-12-09T15:14:58.129 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... ok 2019-12-09T15:15:41.113 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... ok 2019-12-09T15:15:44.736 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:54:36.587 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.751 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:54:36.598 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:54:36.577 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.757 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:54:36.582 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.762 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:54:36.585 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.765 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:54:36.590 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.946 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:54:36.771 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:44.980 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:54:36.804 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:45.019 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:54:36.870 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:15:54.531 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... ok 2019-12-09T15:16:02.641 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... ok 2019-12-09T15:16:18.659 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:55:10.510 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.665 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:55:10.516 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.689 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:55:10.513 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.690 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:55:10.515 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.698 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:55:10.548 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.724 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:55:10.548 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.746 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:55:10.560 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.767 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:55:10.591 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:18.773 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:55:10.597 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:16:27.822 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... ok 2019-12-09T15:16:31.122 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... ok 2019-12-09T15:16:48.846 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... ok 2019-12-09T15:16:55.331 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... ok 2019-12-09T15:17:01.827 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... ok 2019-12-09T15:17:06.613 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... ok 2019-12-09T15:17:10.269 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... ok 2019-12-09T15:17:18.433 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... ok 2019-12-09T15:20:56.347 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:59:48.198 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.358 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:59:48.183 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.362 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:59:48.187 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.369 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:59:48.194 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.374 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:59:48.199 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.375 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:59:48.226 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.531 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:03:59:48.355 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.573 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:03:59:48.424 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:20:56.602 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:03:59:48.427 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:22:49.605 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... ok 2019-12-09T15:23:38.171 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... ok 2019-12-09T15:23:48.686 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... ok 2019-12-09T15:24:07.471 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... ok 2019-12-09T15:24:24.804 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... ok 2019-12-09T15:24:40.530 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:03:32.381 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.534 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:03:32.385 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.568 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:03:32.393 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.575 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:03:32.400 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.595 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:03:32.420 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.596 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:03:32.421 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.661 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:03:32.510 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.676 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:03:32.501 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:24:40.730 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:03:32.552 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.965 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:04:58.815 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.973 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:04:58.824 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.981 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:04:58.805 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.983 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:04:58.808 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.987 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:04:58.812 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:06.996 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:04:58.821 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:07.065 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:04:58.916 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:07.066 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:04:58.891 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:07.067 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:04:58.892 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:26:35.659 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... ok 2019-12-09T15:28:44.040 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... ok 2019-12-09T15:31:19.793 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:10:11.644 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.799 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:10:11.650 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.808 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:10:11.633 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.811 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:10:11.636 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.814 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:10:11.638 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.818 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:10:11.642 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.978 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:10:11.802 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.989 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:10:11.840 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:31:19.990 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:10:11.814 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.301 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:11:54.152 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.304 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:11:54.155 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.362 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:11:54.187 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.364 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:11:54.215 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.367 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:11:54.192 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.379 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:11:54.203 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.380 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:11:54.205 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.429 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:11:54.254 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:33:02.433 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:11:54.258 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.302 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:15:21.153 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.312 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:15:21.158 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.320 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:15:21.144 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.321 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:15:21.146 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.325 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:15:21.149 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.328 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:15:21.153 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.428 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:15:21.279 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.457 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:15:21.278 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:29.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:15:21.312 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:36:32.222 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... ok 2019-12-09T15:39:43.705 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... ok 2019-12-09T15:40:28.302 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... ok 2019-12-09T15:41:13.787 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... ok 2019-12-09T15:41:24.368 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:16.218 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.402 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:16.227 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.421 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:16.271 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.437 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:16.261 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.444 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:16.265 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.465 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:16.290 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.562 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:16.413 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.577 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:16.402 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:24.580 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:16.405 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:39.807 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... ok 2019-12-09T15:41:41.106 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:32.957 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.113 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:32.964 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.118 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:32.942 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.125 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:32.943 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.125 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:32.948 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.125 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:32.947 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.153 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:20:32.978 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.171 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:20:32.996 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:41:41.181 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:20:33.032 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:42:13.544 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... ok 2019-12-09T15:42:20.977 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... ok 2019-12-09T15:42:47.310 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... ok 2019-12-09T15:43:09.566 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... ok 2019-12-09T15:43:16.555 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... Exception in thread Thread-52: 2019-12-09T15:43:16.555 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:16.555 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:16.555 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:16.555 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:16.556 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:16.557 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:16.557 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:16.838 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-54: 2019-12-09T15:43:16.838 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:16.838 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:16.838 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:16.839 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:16.840 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:16.840 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:16.840 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:17.305 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-51: 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:17.306 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:17.307 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:17.410 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-55: 2019-12-09T15:43:17.410 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:17.410 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:17.410 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:17.411 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:17.411 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:17.411 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:17.411 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:17.412 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:17.412 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:17.412 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:17.412 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:17.412 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:17.413 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:26.900 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-62: 2019-12-09T15:43:26.900 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:26.900 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:26.900 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:26.900 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:26.901 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:26.902 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:26.902 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:26.956 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-65: 2019-12-09T15:43:26.956 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:26.956 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:26.956 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:26.957 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:26.958 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:26.958 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:26.958 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:26.958 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:27.079 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-63: 2019-12-09T15:43:27.079 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:27.079 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:27.079 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:27.080 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:27.081 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:27.081 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:27.081 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:39.642 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-71: 2019-12-09T15:43:39.642 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:39.642 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:39.642 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:39.643 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:39.643 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:39.643 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:39.643 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:39.643 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:39.644 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:39.644 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:39.644 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:39.644 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:39.645 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:39.991 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-74: 2019-12-09T15:43:39.992 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:39.992 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:39.992 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:39.992 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:39.993 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:39.993 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:39.993 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:39.993 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:39.994 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:39.994 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:39.994 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:39.994 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:39.995 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:40.312 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-75: 2019-12-09T15:43:40.312 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:40.312 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:40.313 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:40.313 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:40.313 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:40.313 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:40.314 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:40.479 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-73: 2019-12-09T15:43:40.479 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:40.479 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:40.479 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:40.480 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:40.481 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:40.481 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:40.481 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:40.481 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:52.648 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-82: 2019-12-09T15:43:52.648 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:52.648 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:52.648 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:52.648 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:52.649 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:52.650 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:52.650 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:53.122 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-83: 2019-12-09T15:43:53.122 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:53.122 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:53.122 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:53.123 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:53.123 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:53.123 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:53.123 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:53.124 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:53.124 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:53.124 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:53.124 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:53.124 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:53.125 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:53.214 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-81: 2019-12-09T15:43:53.214 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:53.214 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:53.215 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:53.216 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:53.216 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:53.216 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:53.216 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:43:53.246 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-84: 2019-12-09T15:43:53.246 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:43:53.246 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:43:53.246 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:43:53.246 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:43:53.247 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:43:53.248 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:43:53.248 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:43:53.248 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-93: 2019-12-09T15:44:04.470 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:04.470 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:04.470 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:04.471 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:04.472 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:04.472 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:04.472 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:04.472 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:04.472 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-94: 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:04.825 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:04.826 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:04.827 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:04.827 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:04.887 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-92: 2019-12-09T15:44:04.887 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:04.887 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:04.887 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:04.888 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:04.889 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:04.889 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:04.889 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:04.889 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:05.001 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-91: 2019-12-09T15:44:05.001 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:05.001 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:05.001 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:05.002 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:05.003 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:05.003 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:05.003 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:05.003 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:11.990 INFO:teuthology.orchestra.run.smithi174.stderr:ok 2019-12-09T15:44:20.080 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... Exception in thread Thread-104: 2019-12-09T15:44:20.080 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.081 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.082 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.082 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.082 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.082 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.082 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.084 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-106: 2019-12-09T15:44:20.084 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.084 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.084 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.085 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.086 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.086 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.086 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.086 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-103: 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.229 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.231 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.231 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.231 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.232 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.232 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.232 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.232 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.232 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.276 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-101: 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.277 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.278 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.909 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-107: 2019-12-09T15:44:20.909 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.910 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.911 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.911 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.911 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.911 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.911 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.984 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-105: 2019-12-09T15:44:20.985 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.985 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.985 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.985 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.985 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.986 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.987 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.987 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:20.993 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-108: 2019-12-09T15:44:20.993 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:20.993 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:20.993 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:20.994 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:20.994 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:20.994 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:20.994 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:20.994 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:20.995 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:20.995 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:20.995 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:20.995 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:20.996 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:21.159 INFO:teuthology.orchestra.run.smithi174.stderr:Exception in thread Thread-109: 2019-12-09T15:44:21.159 INFO:teuthology.orchestra.run.smithi174.stderr:Traceback (most recent call last): 2019-12-09T15:44:21.159 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner 2019-12-09T15:44:21.159 INFO:teuthology.orchestra.run.smithi174.stderr: self.run() 2019-12-09T15:44:21.159 INFO:teuthology.orchestra.run.smithi174.stderr: File "/usr/lib64/python2.7/threading.py", line 765, in run 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: self.__target(*self.__args, **self.__kwargs) 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/s3tests_boto3/functional/test_s3.py", line 8575, in _do_create_object 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: client.put_object(Bucket=bucket_name, Key=key, Body=body) 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 272, in _api_call 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: return self._make_api_call(operation_name, kwargs) 2019-12-09T15:44:21.160 INFO:teuthology.orchestra.run.smithi174.stderr: File "/home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/botocore/client.py", line 576, in _make_api_call 2019-12-09T15:44:21.161 INFO:teuthology.orchestra.run.smithi174.stderr: raise error_class(parsed_response, operation_name) 2019-12-09T15:44:21.161 INFO:teuthology.orchestra.run.smithi174.stderr:ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: Unknown 2019-12-09T15:44:21.161 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T15:44:36.287 INFO:teuthology.orchestra.run.smithi174.stderr:ok 2019-12-09T15:44:42.620 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... ok 2019-12-09T15:44:46.941 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... ok 2019-12-09T15:44:51.972 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... ok 2019-12-09T15:44:55.754 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... ok 2019-12-09T15:44:59.186 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... ok 2019-12-09T15:45:02.503 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... ok 2019-12-09T15:45:08.262 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... ok 2019-12-09T15:45:11.435 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... ok 2019-12-09T15:45:17.158 INFO:teuthology.orchestra.run.smithi174.stderr:Check for valid x-amz-expiration header after PUT ... ok 2019-12-09T15:45:25.016 INFO:teuthology.orchestra.run.smithi174.stderr:Check for valid x-amz-expiration header on HEAD request ... ok 2019-12-09T15:45:32.388 INFO:teuthology.orchestra.run.smithi174.stderr:Check for valid x-amz-expiration header on GET request ... ok 2019-12-09T15:45:46.090 INFO:teuthology.orchestra.run.smithi174.stderr:Check x-amz-expiration-header object tags interaction ... ok 2019-12-09T15:45:54.864 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... ok 2019-12-09T15:46:00.097 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... ok 2019-12-09T15:46:04.522 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... ok 2019-12-09T15:46:09.035 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... ok 2019-12-09T15:46:15.158 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... ok 2019-12-09T15:46:19.994 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... ok 2019-12-09T15:46:20.729 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:12.580 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.740 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:12.591 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.797 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:12.622 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.813 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:12.637 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.813 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:12.637 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.814 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:12.639 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.977 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:12.802 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:20.978 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:12.802 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:21.033 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:12.884 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:24.133 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... ok 2019-12-09T15:46:30.627 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... ok 2019-12-09T15:46:34.938 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... ok 2019-12-09T15:46:40.205 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok 2019-12-09T15:46:45.787 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok 2019-12-09T15:46:50.422 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok 2019-12-09T15:46:52.044 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:43.895 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.057 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:43.882 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.058 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:43.883 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.060 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:43.911 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.061 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:43.886 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.064 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:43.889 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.106 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:25:43.957 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.114 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:25:43.939 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.115 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:25:43.940 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:46:52.642 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok 2019-12-09T15:46:54.669 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok 2019-12-09T15:46:57.576 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok 2019-12-09T15:47:00.305 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok 2019-12-09T15:49:15.829 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok 2019-12-09T15:49:46.743 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:28:38.594 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.745 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:28:38.596 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.750 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:28:38.574 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.773 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:28:38.597 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.805 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:28:38.630 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.816 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:28:38.640 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.842 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:28:38.693 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.881 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:28:38.705 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:49:46.887 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:28:38.712 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:50:11.637 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok 2019-12-09T15:50:18.102 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok 2019-12-09T15:50:23.856 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... ok 2019-12-09T15:50:30.149 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... ok 2019-12-09T15:50:33.524 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... ok 2019-12-09T15:50:36.372 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... ok 2019-12-09T15:52:02.350 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:30:54.200 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.362 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:30:54.186 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.363 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:30:54.214 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.367 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:30:54.192 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.369 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:30:54.193 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.372 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:30:54.197 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.489 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:30:54.301 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.498 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:30:54.349 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:02.508 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:30:54.333 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:52:47.859 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... ok 2019-12-09T15:53:34.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... ok 2019-12-09T15:54:18.833 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... ok 2019-12-09T15:54:22.856 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... ok 2019-12-09T15:54:27.866 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... ok 2019-12-09T15:54:32.999 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... ok 2019-12-09T15:54:38.779 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... ok 2019-12-09T15:54:42.164 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... ok 2019-12-09T15:54:47.173 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... ok 2019-12-09T15:54:53.938 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... ok 2019-12-09T15:55:02.035 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... ok 2019-12-09T15:55:09.685 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... ok 2019-12-09T15:55:18.723 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... ok 2019-12-09T15:55:30.032 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... ok 2019-12-09T15:55:44.234 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... ok 2019-12-09T15:55:57.236 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... ok 2019-12-09T15:56:09.886 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... ok 2019-12-09T15:56:24.079 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... ok 2019-12-09T15:56:33.148 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... ok 2019-12-09T15:56:44.347 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... ok 2019-12-09T15:56:55.890 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... ok 2019-12-09T15:57:04.882 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... ok 2019-12-09T15:57:15.946 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:36:07.796 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:15.950 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:36:07.801 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:15.952 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:36:07.777 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:15.955 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:36:07.779 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:15.957 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:36:07.781 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:15.958 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:36:07.783 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:16.171 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:36:08.022 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:16.215 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:36:08.040 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:16.220 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:36:08.045 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:57:16.342 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... ok 2019-12-09T15:57:27.855 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... ok 2019-12-09T15:57:33.825 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... ok 2019-12-09T15:57:38.837 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... ok 2019-12-09T15:57:44.475 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... ok 2019-12-09T15:57:58.410 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... ok 2019-12-09T15:58:08.588 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:37:00.439 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.610 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:37:00.435 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.612 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:37:00.463 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.616 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:37:00.441 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.616 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:37:00.441 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.660 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:37:00.485 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.665 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:37:00.489 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.683 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:37:00.534 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:08.705 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:37:00.529 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T15:58:10.756 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... ok 2019-12-09T15:58:23.678 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... ok 2019-12-09T15:58:44.029 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... ok 2019-12-09T16:01:05.778 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... ok 2019-12-09T16:01:17.416 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... ok 2019-12-09T16:01:32.575 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... ok 2019-12-09T16:01:46.541 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... ok 2019-12-09T16:02:05.512 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... ok 2019-12-09T16:02:20.454 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... ok 2019-12-09T16:02:26.271 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:41:18.122 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.294 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:41:18.145 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.308 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:41:18.133 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.311 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:41:18.136 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.312 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:41:18.137 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.319 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:41:18.144 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.470 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:41:18.320 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.498 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:41:18.323 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.499 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:41:18.324 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:02:26.509 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... ok 2019-12-09T16:02:38.040 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... ok 2019-12-09T16:02:52.764 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... ok 2019-12-09T16:02:58.278 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... ok 2019-12-09T16:03:02.147 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... ok 2019-12-09T16:03:06.815 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... ok 2019-12-09T16:03:10.059 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... ok 2019-12-09T16:03:13.379 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... ok 2019-12-09T16:03:16.396 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... ok 2019-12-09T16:03:19.989 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... ok 2019-12-09T16:03:22.424 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... ok 2019-12-09T16:03:24.924 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... ok 2019-12-09T16:03:39.195 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... ok 2019-12-09T16:03:42.857 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... ok 2019-12-09T16:03:50.954 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... ok 2019-12-09T16:04:03.485 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... ok 2019-12-09T16:04:07.580 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... ok 2019-12-09T16:04:19.870 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... ok 2019-12-09T16:04:32.004 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... ok 2019-12-09T16:04:45.202 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... ok 2019-12-09T16:04:55.299 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... ok 2019-12-09T16:05:08.390 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... ok 2019-12-09T16:05:23.992 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... ok 2019-12-09T16:05:33.305 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... ok 2019-12-09T16:05:38.255 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... ok 2019-12-09T16:05:47.445 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... ok 2019-12-09T16:05:58.012 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... ok 2019-12-09T16:06:03.074 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... ok 2019-12-09T16:06:12.831 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... ok 2019-12-09T16:06:24.146 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... ok 2019-12-09T16:06:30.609 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:45:22.460 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.616 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:45:22.467 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.619 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:45:22.444 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.623 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:45:22.448 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.631 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:45:22.456 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.633 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:45:22.458 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.736 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:45:22.561 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.747 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:45:22.572 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:30.838 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:45:22.689 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:06:39.517 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... ok 2019-12-09T16:06:52.141 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... ok 2019-12-09T16:06:52.157 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-12-09T16:07:38.352 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:46:30.202 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.361 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:46:30.212 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.368 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:46:30.192 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.373 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:46:30.197 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.375 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:46:30.200 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.378 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:46:30.203 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.425 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:46:30.276 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.427 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:46:30.252 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:07:38.435 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:46:30.260 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.628 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:51:41.479 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.636 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:51:41.487 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.646 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:51:41.470 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.646 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:51:41.471 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.651 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:51:41.474 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.653 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:51:41.477 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.819 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:51:41.670 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.853 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:51:41.678 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:12:49.864 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:51:41.689 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.466 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:53:44.317 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.477 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:53:44.328 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.539 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:53:44.364 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.545 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:53:44.370 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.548 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:53:44.373 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.552 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:53:44.376 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.580 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:53:44.431 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.616 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:53:44.441 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:14:52.636 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:53:44.461 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.809 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:56:52.660 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.821 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:56:52.672 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.837 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:56:52.661 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.843 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:56:52.668 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.849 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:56:52.673 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.853 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:56:52.678 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.944 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:04:56:52.795 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.977 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:04:56:52.801 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:18:00.977 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:04:56:52.802 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.045 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:01.895 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.052 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:01.903 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.053 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:01.876 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.053 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:01.877 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.057 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:01.881 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.058 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:01.883 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.233 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:02.057 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.251 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:02.075 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:10.253 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:02.104 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.363 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:06.214 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.369 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:06.220 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.392 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:02:06.243 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.394 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:06.219 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.399 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:06.224 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.420 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:06.244 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.440 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:06.265 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.461 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:02:06.286 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:23:14.490 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:02:06.315 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.489 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:05:52.339 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.491 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:05:52.342 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.551 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:05:52.375 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.570 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:05:52.394 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.595 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:05:52.420 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.620 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:05:52.444 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.639 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:05:52.490 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.732 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:05:52.557 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:27:00.763 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:05:52.588 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.953 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:07:12.804 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.963 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:07:12.814 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.968 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:07:12.793 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.973 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:07:12.798 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.976 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:07:12.800 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:20.980 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:07:12.805 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:21.049 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:07:12.874 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:21.083 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:07:12.907 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:28:21.096 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:07:12.946 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.588 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:10:28.439 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.596 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:10:28.443 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.634 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:10:28.458 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.653 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:10:28.477 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.663 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:10:28.488 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.684 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:10:28.505 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.720 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:10:28.570 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.769 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:10:28.593 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:31:36.785 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:10:28.609 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.042 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:12:23.893 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.045 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:12:23.896 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.054 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:12:23.879 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.059 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:12:23.884 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.060 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:12:23.885 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.063 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:12:23.888 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.142 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:12:23.967 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.149 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:12:23.974 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:33:32.150 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:12:24.002 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.171 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:17:35.022 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.181 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:17:35.032 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.199 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:17:35.023 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.202 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:17:35.027 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.208 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:17:35.033 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.209 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:17:35.034 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.397 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:17:35.222 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.419 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:17:35.244 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:38:43.455 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:17:35.306 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.408 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:18:50.259 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.413 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:18:50.264 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.435 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:18:50.260 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.453 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:18:50.277 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.453 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:18:50.278 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.456 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:18:50.281 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.492 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:18:50.343 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.497 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:18:50.322 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:39:58.498 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:18:50.323 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.012 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:22:45.863 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.019 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:22:45.843 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.022 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:22:45.847 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.031 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:22:45.856 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:22:45.860 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.036 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:22:45.887 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.181 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:22:46.006 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.223 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:22:46.058 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:43:54.223 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:22:46.016 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.674 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:12.525 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.679 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:12.531 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.718 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:12.543 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.721 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:12.546 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.726 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:12.551 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.749 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:12.574 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.797 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:12.649 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.848 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:12.672 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:48:20.862 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:12.687 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.023 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:57.874 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.029 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:57.880 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.035 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:57.860 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.040 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:57.865 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.050 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:57.875 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.062 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:57.887 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.114 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:27:57.965 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.136 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:27:57.961 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:49:06.150 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:27:57.975 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.479 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:33:10.330 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.490 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:33:10.342 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.496 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:33:10.321 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.500 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:33:10.325 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.503 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:33:10.328 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.508 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:33:10.333 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.676 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:33:10.501 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.698 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:33:10.523 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:54:18.719 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:33:10.571 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.745 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:38:22.596 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.772 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:38:22.586 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.772 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:38:22.594 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.772 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:38:22.588 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.772 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:38:22.620 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.773 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:38:22.599 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.954 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:38:22.779 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:30.988 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:38:22.813 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T16:59:31.059 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:38:22.911 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:40.985 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:32.836 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.007 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:32.826 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.007 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:32.830 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.007 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:32.829 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.008 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:32.850 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.009 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:32.835 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.211 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:33.036 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.227 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:33.052 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:04:41.253 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:33.105 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.300 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:56.152 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.356 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:56.207 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.357 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:56.182 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.359 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:56.185 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.394 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:43:56.245 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.397 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:43:56.222 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.413 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:56.238 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.414 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:56.240 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:05:04.436 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:43:56.261 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.253 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:46:37.105 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.259 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:46:37.110 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.308 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:46:37.133 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.311 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:46:37.136 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.322 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:46:37.147 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.324 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:46:37.149 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.376 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:46:37.227 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.431 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:46:37.256 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:07:45.433 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:46:37.258 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.779 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:48:43.630 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.791 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:48:43.643 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.805 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:48:43.630 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.807 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:48:43.633 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.808 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:48:43.634 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.812 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:48:43.638 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.896 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:48:43.748 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.905 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:48:43.730 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:09:51.920 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:48:43.745 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.589 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:52:18.441 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.598 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:52:18.450 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.632 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:52:18.457 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.638 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:52:18.464 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.641 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:52:18.467 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.649 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:52:18.474 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.725 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:52:18.577 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.738 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:52:18.564 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:13:26.823 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:52:18.648 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.030 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:53:53.882 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.045 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:53:53.897 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.058 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:53:53.883 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.060 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:53:53.886 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.063 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:53:53.889 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.065 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:53:53.890 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.127 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:53:53.979 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.140 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:53:53.965 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:15:02.145 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:53:53.970 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.508 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:59:09.360 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.513 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:59:09.365 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.520 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:59:09.346 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.522 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:59:09.348 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.524 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:59:09.350 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.526 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:59:09.352 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.705 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:05:59:09.531 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.714 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:05:59:09.566 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:20:17.743 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:05:59:09.568 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.540 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:00:40.392 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.545 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:00:40.395 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.578 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:00:40.403 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.582 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:00:40.408 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.584 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:00:40.410 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.591 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:00:40.416 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.616 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:00:40.468 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.631 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:00:40.457 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:21:48.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:00:40.484 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.671 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:04:19.522 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.698 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:04:19.508 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.698 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:04:19.512 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.698 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:04:19.511 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.698 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:04:19.515 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.698 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:04:19.532 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.816 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:04:19.642 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.821 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:04:19.647 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:25:27.862 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:04:19.713 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.375 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:07.227 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.409 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:07.260 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.453 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:07.279 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.469 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:07.295 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.476 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:07.302 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.496 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:07.322 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.576 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:07.427 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.613 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:07.439 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:15.615 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:07.441 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.276 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:31.128 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.280 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:31.132 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.298 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:31.124 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.301 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:31.127 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.336 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:09:31.162 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.338 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:31.164 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.344 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:31.170 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.365 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:09:31.217 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:30:39.398 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:09:31.224 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.722 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:14:44.574 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:14:44.560 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:14:44.568 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.752 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:14:44.567 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.753 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:14:44.573 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.753 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:14:44.583 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.929 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:14:44.755 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.942 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:14:44.768 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:35:52.947 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:14:44.799 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.249 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:17:29.100 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.271 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:17:29.104 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.306 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:17:29.131 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.307 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:17:29.133 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.386 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:17:29.212 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.387 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:17:29.213 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.399 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:17:29.251 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.402 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:17:29.228 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:38:37.495 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:17:29.321 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.831 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:19:54.683 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.840 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:19:54.692 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.847 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:19:54.673 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.848 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:19:54.674 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.851 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:19:54.676 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.855 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:19:54.681 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.934 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:19:54.786 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.958 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:19:54.784 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:41:02.980 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:19:54.806 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.662 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:08.513 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:08.496 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.676 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:08.501 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.679 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:08.530 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.679 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:08.505 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.686 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:08.511 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.924 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:08.774 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.937 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:08.762 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:16.966 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:08.792 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.121 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:50.972 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.172 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:50.997 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.172 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:51.023 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.192 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:51.012 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.218 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:25:51.069 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.238 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:25:51.062 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.243 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:51.067 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.243 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:51.069 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:46:59.272 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:25:51.098 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:54.927 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:27:46.778 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:54.943 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:27:46.794 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.009 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:27:46.835 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.011 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:27:46.836 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.025 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:27:46.850 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.030 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:27:46.881 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.030 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:27:46.855 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.095 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:27:46.920 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:48:55.114 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:27:46.939 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.901 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:30:18.751 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.912 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:30:18.737 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.912 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:30:18.763 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.918 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:30:18.740 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.919 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:30:18.741 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:26.919 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:30:18.744 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:27.023 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:30:18.848 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:27.033 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:30:18.858 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:51:27.034 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:30:18.885 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:54:23.589 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-12-09T17:54:23.610 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-12-09T17:54:23.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-12-09T17:54:23.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-12-09T17:54:23.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-12-09T17:54:23.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-12-09T17:54:23.611 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-12-09T17:54:23.972 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-12-09T17:54:24.153 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-12-09T17:54:24.507 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-12-09T17:54:24.532 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-12-09T17:54:24.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-12-09T17:54:24.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-12-09T17:54:24.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-12-09T17:54:24.558 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-12-09T17:54:24.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-12-09T17:54:24.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-12-09T17:54:24.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-12-09T17:54:24.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-12-09T17:54:24.559 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-12-09T17:54:24.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-12-09T17:54:24.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-12-09T17:54:24.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-12-09T17:54:24.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-12-09T17:54:24.561 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-12-09T17:54:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-12-09T17:54:24.562 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-12-09T17:54:24.563 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-12-09T17:54:24.563 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-12-09T17:54:24.563 INFO:teuthology.orchestra.run.smithi174.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-12-09T17:54:24.586 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-12-09T17:54:24.607 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-12-09T17:54:24.608 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-12-09T17:54:24.754 INFO:teuthology.orchestra.run.smithi174.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-12-09T17:54:24.754 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T17:54:24.754 INFO:teuthology.orchestra.run.smithi174.stderr:---------------------------------------------------------------------- 2019-12-09T17:54:24.754 INFO:teuthology.orchestra.run.smithi174.stderr:Ran 611 tests in 14654.515s 2019-12-09T17:54:24.755 INFO:teuthology.orchestra.run.smithi174.stderr: 2019-12-09T17:54:24.755 INFO:teuthology.orchestra.run.smithi174.stderr:OK (SKIP=75) 2019-12-09T17:54:24.805 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-12-09T17:54:24.805 INFO:teuthology.orchestra.run.smithi174:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-12-09T17:54:24.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-12-09T17:54:24.923 INFO:tasks.s3tests:Cleaning up boto... 2019-12-09T17:54:24.923 INFO:teuthology.orchestra.run.smithi174:> rm /home/ubuntu/cephtest/boto.cfg 2019-12-09T17:54:24.965 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-12-09T17:54:25.190 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.188+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.228 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.226+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.228 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.226+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.233+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.238 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.233+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.245+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.247 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.245+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.268 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.265+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:25.453 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.451+0000 7feeed844ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T17:54:25.454 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.451+0000 7feeed844ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T17:54:25.454 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.451+0000 7feeed844ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:54:25.454 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.451+0000 7feeed844ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:54:25.454 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.451+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.459+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:25.461 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.459+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.468 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.466+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:25.469 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.467+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.475+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:25.477 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.475+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.486 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.484+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:25.486 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.484+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.493 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.491+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.491+0000 7feeed844ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:54:25.494 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.491+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.499+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.502 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.499+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.511+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:25.513 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.511+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.524+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:25.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.524+0000 7feeed844ec0 20 zonegroup default 2019-12-09T17:54:25.526 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.524+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.532 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.531+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.531+0000 7feeed844ec0 10 Cannot find current period zone using local zone 2019-12-09T17:54:25.533 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.531+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.538 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.536+0000 7feeed844ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:25.538 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.536+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.545+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:25.547 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.545+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:25.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.556+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:25.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.556+0000 7feeed844ec0 20 zone default found 2019-12-09T17:54:25.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.556+0000 7feeed844ec0 20 started sync module instance, tier type = 2019-12-09T17:54:25.558 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.556+0000 7feeed844ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T17:54:25.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=0 2019-12-09T17:54:25.899 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=1 2019-12-09T17:54:25.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=2 2019-12-09T17:54:25.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=3 2019-12-09T17:54:25.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=4 2019-12-09T17:54:25.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=5 2019-12-09T17:54:25.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.897+0000 7feeed844ec0 20 add_watcher() i=6 2019-12-09T17:54:25.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.930+0000 7feeed844ec0 20 add_watcher() i=7 2019-12-09T17:54:25.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.930+0000 7feeed844ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:54:25.933 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.931+0000 7feeab7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:25.946 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.944+0000 7feedb26f700 0 -- 172.21.15.174:0/2515405581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fee1c002850 msgr2=0x7fee1c002e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T17:54:25.991 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:25.989+0000 7feedaa6e700 0 -- 172.21.15.174:0/415436079 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x555daac3d780 msgr2=0x555daac3fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:27.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.640+0000 7fee19ffb700 20 reqs_thread_entry: start 2019-12-09T17:54:27.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.641+0000 7fee137fe700 20 reqs_thread_entry: start 2019-12-09T17:54:27.643 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.641+0000 7feeed844ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:54:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.641+0000 7fee11ffb700 20 reqs_thread_entry: start 2019-12-09T17:54:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.641+0000 7feeed844ec0 20 get_system_obj_state: rctx=0x7ffe5d9131a8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x555daaca6f90 s->prefetch_data=0 2019-12-09T17:54:27.644 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.641+0000 7feeed844ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T17:54:27.658 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.655+0000 7feeed844ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-12-09T17:54:27.658 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.656+0000 7feeed844ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:27.658 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.656+0000 7feeed844ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:54:27.658 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.656+0000 7feeed844ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:54:27.658 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.656+0000 7feeed844ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:54:27.659 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.656+0000 7feeed844ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:27.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.675+0000 7feeed844ec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T17:54:27.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.675+0000 7feeed844ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-12-09T17:54:27.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.675+0000 7feeed844ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:27.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.691+0000 7feeed844ec0 10 removing key index: TAZOQBUSHGUKLFCLQYQR 2019-12-09T17:54:27.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.691+0000 7feeed844ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-12-09T17:54:27.709 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:27.707+0000 7feebbfff700 10 RGWWatcher::handle_notify() notify_id 592705487372 cookie 93860785118912 notifier 5020 bl.length()=170 2019-12-09T17:54:28.541 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:28.539+0000 7feeed844ec0 10 removing email index: bar.client.0+test@test.test 2019-12-09T17:54:28.542 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:28.539+0000 7feeed844ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-12-09T17:54:28.561 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:28.558+0000 7feebbfff700 10 RGWWatcher::handle_notify() notify_id 592705487373 cookie 93860785113056 notifier 5020 bl.length()=178 2019-12-09T17:54:28.993 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:28.991+0000 7feeed844ec0 10 removing user buckets index 2019-12-09T17:54:28.994 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:28.991+0000 7feeed844ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-12-09T17:54:29.008 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.006+0000 7feebbfff700 10 RGWWatcher::handle_notify() notify_id 592705486926 cookie 93860785114928 notifier 5020 bl.length()=169 2019-12-09T17:54:29.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.083+0000 7feeed844ec0 10 removing user index: bar.client.0 2019-12-09T17:54:29.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.083+0000 7feeed844ec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-12-09T17:54:29.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.083+0000 7feeed844ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-12-09T17:54:29.094 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.092+0000 7feebbfff700 10 RGWWatcher::handle_notify() notify_id 592705487374 cookie 93860785113056 notifier 5020 bl.length()=161 2019-12-09T17:54:29.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.675+0000 7feeed844ec0 20 remove_watcher() i=0 2019-12-09T17:54:29.677 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.675+0000 7feeed844ec0 2 removed watcher, disabling cache 2019-12-09T17:54:29.746 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.743+0000 7feeed844ec0 20 remove_watcher() i=1 2019-12-09T17:54:29.831 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.828+0000 7feeed844ec0 20 remove_watcher() i=2 2019-12-09T17:54:29.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.860+0000 7feeed844ec0 20 remove_watcher() i=3 2019-12-09T17:54:29.918 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.915+0000 7feeed844ec0 20 remove_watcher() i=4 2019-12-09T17:54:29.954 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.951+0000 7feeed844ec0 20 remove_watcher() i=5 2019-12-09T17:54:29.998 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:29.995+0000 7feeed844ec0 20 remove_watcher() i=6 2019-12-09T17:54:30.031 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.028+0000 7feeed844ec0 20 remove_watcher() i=7 2019-12-09T17:54:30.040 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-12-09T17:54:30.167 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.164+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.192+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.195 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.192+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.197+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.200 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.197+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.208 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.205+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.208 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.205+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.222 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.219+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:30.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.384+0000 7f214f597ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T17:54:30.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.384+0000 7f214f597ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T17:54:30.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.384+0000 7f214f597ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:54:30.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.384+0000 7f214f597ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:54:30.387 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.384+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.398+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:30.401 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.398+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.409+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:30.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.409+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.424+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:30.427 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.424+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.441 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.436+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:30.441 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.436+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.449 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.446+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.446+0000 7f214f597ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:54:30.450 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.446+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.457 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.454+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.457 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.454+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.481 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.472+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:30.482 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.473+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.484+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.484+0000 7f214f597ec0 20 zonegroup default 2019-12-09T17:54:30.487 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.484+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.492+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.492+0000 7f214f597ec0 10 Cannot find current period zone using local zone 2019-12-09T17:54:30.495 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.492+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.502+0000 7f214f597ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:30.505 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.502+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.522 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.519+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:30.522 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.519+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:30.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.531+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:30.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.531+0000 7f214f597ec0 20 zone default found 2019-12-09T17:54:30.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.531+0000 7f214f597ec0 20 started sync module instance, tier type = 2019-12-09T17:54:30.535 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.531+0000 7f214f597ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T17:54:30.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=0 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=1 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=2 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=3 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=4 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=5 2019-12-09T17:54:30.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.918+0000 7f214f597ec0 20 add_watcher() i=6 2019-12-09T17:54:30.963 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.960+0000 7f214f597ec0 20 add_watcher() i=7 2019-12-09T17:54:30.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.960+0000 7f214f597ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:54:30.986 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.960+0000 7f210cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:30.987 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:30.962+0000 7f213cfc2700 0 -- >> v2:172.21.15.106:3300/0 conn(0x5650256fb440 msgr2=0x5650256fb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T17:54:31.048 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:31.045+0000 7f213c7c1700 0 -- 172.21.15.174:0/3715300157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565025701de0 msgr2=0x565025708700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:32.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.835+0000 7f207effd700 20 reqs_thread_entry: start 2019-12-09T17:54:32.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.836+0000 7f207cff9700 20 reqs_thread_entry: start 2019-12-09T17:54:32.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.836+0000 7f214f597ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:54:32.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.836+0000 7f2072ffd700 20 reqs_thread_entry: start 2019-12-09T17:54:32.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.836+0000 7f214f597ec0 20 get_system_obj_state: rctx=0x7fff3aa2ca28 obj=default.rgw.meta:users.uid:foo.client.0 state=0x565025776830 s->prefetch_data=0 2019-12-09T17:54:32.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.836+0000 7f214f597ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T17:54:32.855 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-12-09T17:54:32.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:32.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:54:32.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:54:32.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:54:32.856 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.852+0000 7f214f597ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:32.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.862+0000 7f214f597ec0 20 rados_obj.operate() r=0 bl.length=446 2019-12-09T17:54:32.865 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.862+0000 7f214f597ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-12-09T17:54:32.866 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.862+0000 7f214f597ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:32.891 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.888+0000 7f214f597ec0 10 removing key index: JBMMFVPQAOCYGFILKQBF 2019-12-09T17:54:32.892 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.888+0000 7f214f597ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-12-09T17:54:32.900 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:32.897+0000 7f2131ffb700 10 RGWWatcher::handle_notify() notify_id 592705486927 cookie 94902225387728 notifier 4703 bl.length()=170 2019-12-09T17:54:33.091 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.083+0000 7f214f597ec0 10 removing email index: foo.client.0+test@test.test 2019-12-09T17:54:33.128 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.083+0000 7f214f597ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-12-09T17:54:33.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.090+0000 7f2131ffb700 10 RGWWatcher::handle_notify() notify_id 592705486898 cookie 94902225397648 notifier 4703 bl.length()=178 2019-12-09T17:54:33.632 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.630+0000 7f214f597ec0 10 removing user buckets index 2019-12-09T17:54:33.633 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.630+0000 7f214f597ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-12-09T17:54:33.648 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:33.645+0000 7f2131ffb700 10 RGWWatcher::handle_notify() notify_id 592705487375 cookie 94902225385312 notifier 4703 bl.length()=169 2019-12-09T17:54:34.129 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.126+0000 7f214f597ec0 10 removing user index: foo.client.0 2019-12-09T17:54:34.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.126+0000 7f214f597ec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-12-09T17:54:34.130 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.126+0000 7f214f597ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-12-09T17:54:34.149 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.144+0000 7f2131ffb700 10 RGWWatcher::handle_notify() notify_id 592705487494 cookie 94902225383328 notifier 4703 bl.length()=161 2019-12-09T17:54:34.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.973+0000 7f214f597ec0 20 remove_watcher() i=0 2019-12-09T17:54:34.976 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:34.973+0000 7f214f597ec0 2 removed watcher, disabling cache 2019-12-09T17:54:35.042 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.040+0000 7f214f597ec0 20 remove_watcher() i=1 2019-12-09T17:54:35.084 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.081+0000 7f214f597ec0 20 remove_watcher() i=2 2019-12-09T17:54:35.123 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.120+0000 7f214f597ec0 20 remove_watcher() i=3 2019-12-09T17:54:35.160 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.157+0000 7f214f597ec0 20 remove_watcher() i=4 2019-12-09T17:54:35.194 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.191+0000 7f214f597ec0 20 remove_watcher() i=5 2019-12-09T17:54:35.226 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.223+0000 7f214f597ec0 20 remove_watcher() i=6 2019-12-09T17:54:35.261 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.257+0000 7f214f597ec0 20 remove_watcher() i=7 2019-12-09T17:54:35.267 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-12-09T17:54:35.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.381+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.412 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.409+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.413 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.410+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.415+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.415+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.422+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.425 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.422+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.440 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.437+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:35.604 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.601+0000 7f64bd640ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T17:54:35.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.601+0000 7f64bd640ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T17:54:35.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.601+0000 7f64bd640ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:54:35.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.601+0000 7f64bd640ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:54:35.605 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.601+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.621 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.614+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:35.621 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.614+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.629 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.626+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:35.630 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.626+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.651+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:35.654 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.651+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.663 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.660+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:35.664 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.660+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.671+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.671+0000 7f64bd640ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:54:35.674 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.671+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.679+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.682 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.679+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.696 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.693+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:35.697 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.693+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.702+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:35.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.702+0000 7f64bd640ec0 20 zonegroup default 2019-12-09T17:54:35.705 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.702+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.710+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.713 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.710+0000 7f64bd640ec0 10 Cannot find current period zone using local zone 2019-12-09T17:54:35.714 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.710+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.718+0000 7f64bd640ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:35.721 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.718+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.733 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.730+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:35.734 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.730+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:35.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.745+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:35.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.745+0000 7f64bd640ec0 20 zone default found 2019-12-09T17:54:35.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.745+0000 7f64bd640ec0 20 started sync module instance, tier type = 2019-12-09T17:54:35.748 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:35.745+0000 7f64bd640ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T17:54:36.113 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=0 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=1 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=2 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=3 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=4 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=5 2019-12-09T17:54:36.114 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.110+0000 7f64bd640ec0 20 add_watcher() i=6 2019-12-09T17:54:36.147 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.144+0000 7f64bd640ec0 20 add_watcher() i=7 2019-12-09T17:54:36.147 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.144+0000 7f64bd640ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:54:36.147 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.144+0000 7f647b7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:36.152 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.149+0000 7f64aa86a700 0 -- 172.21.15.174:0/1961697190 >> v1:172.21.15.174:6789/0 conn(0x5646c50c6590 legacy=0x5646c50c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:36.194 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:36.191+0000 7f64aa069700 0 -- 172.21.15.174:0/2072239015 >> [v2:172.21.15.106:6820/9715,v1:172.21.15.106:6821/9715] conn(0x5646c50db680 msgr2=0x5646c50ddaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:37.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.808+0000 7f63e9ffb700 20 reqs_thread_entry: start 2019-12-09T17:54:37.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.808+0000 7f63e37fe700 20 reqs_thread_entry: start 2019-12-09T17:54:37.812 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.808+0000 7f64bd640ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:54:37.813 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.809+0000 7f63e1ffb700 20 reqs_thread_entry: start 2019-12-09T17:54:37.813 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.809+0000 7f64bd640ec0 20 get_system_obj_state: rctx=0x7ffe20e51d68 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x5646c5144e20 s->prefetch_data=0 2019-12-09T17:54:37.813 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.809+0000 7f64bd640ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-12-09T17:54:37.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-12-09T17:54:37.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T17:54:37.823 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-12-09T17:54:37.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-12-09T17:54:37.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-12-09T17:54:37.824 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.820+0000 7f64bd640ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:37.834 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.831+0000 7f64bd640ec0 20 rados_obj.operate() r=0 bl.length=505 2019-12-09T17:54:37.835 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.831+0000 7f64bd640ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-12-09T17:54:37.835 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.831+0000 7f64bd640ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-12-09T17:54:37.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.836+0000 7f64bd640ec0 10 removing key index: NLHORHQOCYKKTUEZLGGT 2019-12-09T17:54:37.839 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.836+0000 7f64bd640ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=170 2019-12-09T17:54:37.849 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.846+0000 7f648bfff700 10 RGWWatcher::handle_notify() notify_id 592705486928 cookie 94861953554480 notifier 4715 bl.length()=170 2019-12-09T17:54:37.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.925+0000 7f64bd640ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-12-09T17:54:37.928 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.925+0000 7f64bd640ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-12-09T17:54:37.936 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:37.933+0000 7f648bfff700 10 RGWWatcher::handle_notify() notify_id 592705487495 cookie 94861953560240 notifier 4715 bl.length()=193 2019-12-09T17:54:38.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.360+0000 7f64bd640ec0 10 removing user buckets index 2019-12-09T17:54:38.363 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.360+0000 7f64bd640ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-12-09T17:54:38.370 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.367+0000 7f648bfff700 10 RGWWatcher::handle_notify() notify_id 592705486899 cookie 94861953557968 notifier 4715 bl.length()=184 2019-12-09T17:54:38.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.934+0000 7f64bd640ec0 10 removing user index: testx$tenanteduser.client.0 2019-12-09T17:54:38.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.934+0000 7f64bd640ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-12-09T17:54:38.937 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.934+0000 7f64bd640ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-12-09T17:54:38.956 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:38.953+0000 7f648bfff700 10 RGWWatcher::handle_notify() notify_id 592705487376 cookie 94861953556336 notifier 4715 bl.length()=176 2019-12-09T17:54:39.126 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.123+0000 7f64bd640ec0 20 remove_watcher() i=0 2019-12-09T17:54:39.127 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.123+0000 7f64bd640ec0 2 removed watcher, disabling cache 2019-12-09T17:54:39.179 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.176+0000 7f64bd640ec0 20 remove_watcher() i=1 2019-12-09T17:54:39.217 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.214+0000 7f64bd640ec0 20 remove_watcher() i=2 2019-12-09T17:54:39.257 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.254+0000 7f64bd640ec0 20 remove_watcher() i=3 2019-12-09T17:54:39.311 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.308+0000 7f64bd640ec0 20 remove_watcher() i=4 2019-12-09T17:54:39.353 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.350+0000 7f64bd640ec0 20 remove_watcher() i=5 2019-12-09T17:54:39.388 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.385+0000 7f64bd640ec0 20 remove_watcher() i=6 2019-12-09T17:54:39.416 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:39.413+0000 7f64bd640ec0 20 remove_watcher() i=7 2019-12-09T17:54:39.423 INFO:tasks.s3tests:Removing s3-tests... 2019-12-09T17:54:39.423 INFO:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-12-09T17:54:39.564 INFO:teuthology.run_tasks:Running task swift... 2019-12-09T17:54:39.701 INFO:tasks.swift:clients=['client.0'] 2019-12-09T17:54:39.707 INFO:tasks.swift:Downloading swift... 2019-12-09T17:54:39.708 INFO:teuthology.orchestra.run.smithi174:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-12-09T17:54:39.839 INFO:teuthology.orchestra.run.smithi174.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-12-09T17:54:40.237 INFO:tasks.swift:Creating rgw users... 2019-12-09T17:54:40.237 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret RkbXFH2UeJl+StpxZYDmupMU55Zw2uD5fCBahVbHX73XzaZE6+y9eA== --email foo.client.0+test@test.test --key-type swift --access full 2019-12-09T17:54:40.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.356+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.384 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.381+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.385 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.381+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.392 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.389+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.392 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.389+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.396+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.399 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.396+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.420 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.411+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:40.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.599+0000 7f7f51206ec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T17:54:40.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.599+0000 7f7f51206ec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T17:54:40.602 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.599+0000 7f7f51206ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:54:40.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.599+0000 7f7f51206ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:54:40.603 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.599+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.614 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.611+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:40.615 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.611+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.631 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.627+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:40.631 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.628+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.639+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:40.642 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.639+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.652 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.649+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:40.653 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.650+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.657+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.657+0000 7f7f51206ec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:54:40.660 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.657+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.662+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.665 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.662+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.676 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.673+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:40.676 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.673+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.681+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:40.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.681+0000 7f7f51206ec0 20 zonegroup default 2019-12-09T17:54:40.684 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.681+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.689+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.692 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.689+0000 7f7f51206ec0 10 Cannot find current period zone using local zone 2019-12-09T17:54:40.693 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.689+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.721+0000 7f7f51206ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:40.724 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.721+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.733+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:40.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.733+0000 7f7f51206ec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:40.743 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.740+0000 7f7f51206ec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:40.744 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.740+0000 7f7f51206ec0 20 zone default found 2019-12-09T17:54:40.744 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.740+0000 7f7f51206ec0 20 started sync module instance, tier type = 2019-12-09T17:54:40.744 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:40.740+0000 7f7f51206ec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T17:54:41.085 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=0 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=1 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=2 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=3 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=4 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=5 2019-12-09T17:54:41.086 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.082+0000 7f7f51206ec0 20 add_watcher() i=6 2019-12-09T17:54:41.133 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.130+0000 7f7f51206ec0 20 add_watcher() i=7 2019-12-09T17:54:41.133 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.130+0000 7f7f51206ec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:54:41.133 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.130+0000 7f7f0efdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:41.140 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.137+0000 7f7f3ec31700 0 -- 172.21.15.174:0/2968290253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac8992d610 msgr2=0x55ac8992da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-12-09T17:54:41.343 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.340+0000 7f7f3ec31700 0 --2- 172.21.15.174:0/2968290253 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55ac8992d610 0x55ac8992da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-12-09T17:54:41.410 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:41.407+0000 7f7f3e430700 0 -- 172.21.15.174:0/2680261772 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f7e80022470 msgr2=0x7f7e80024980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:42.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.812+0000 7f7e7d7fa700 20 reqs_thread_entry: start 2019-12-09T17:54:42.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.813+0000 7f7e76ffd700 20 reqs_thread_entry: start 2019-12-09T17:54:42.816 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.813+0000 7f7f51206ec0 20 init_complete bucket index max shards: 0 2019-12-09T17:54:42.817 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.813+0000 7f7e757fa700 20 reqs_thread_entry: start 2019-12-09T17:54:42.817 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.813+0000 7f7f51206ec0 20 get_system_obj_state: rctx=0x7ffdf91de0b8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55ac899b12e0 s->prefetch_data=0 2019-12-09T17:54:42.817 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.813+0000 7f7f51206ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-12-09T17:54:42.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.824+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-12-09T17:54:42.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.824+0000 7f7f51206ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:42.840 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.824+0000 7f7f51206ec0 20 get_system_obj_state: rctx=0x7ffdf91de0c8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x55ac899b3070 s->prefetch_data=0 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.824+0000 7f7f51206ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.830+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.830+0000 7f7f51206ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.830+0000 7f7f51206ec0 20 get_system_obj_state: rctx=0x7ffdf91ddcd8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55ac899b3070 s->prefetch_data=0 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.830+0000 7f7f51206ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-12-09T17:54:42.841 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.836+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T17:54:42.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.836+0000 7f7f51206ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:54:42.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.836+0000 7f7f51206ec0 20 get_system_obj_state: rctx=0x7ffdf91dde58 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x55ac899b3320 s->prefetch_data=0 2019-12-09T17:54:42.842 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.836+0000 7f7f51206ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-12-09T17:54:42.848 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.841+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-12-09T17:54:42.848 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.841+0000 7f7f51206ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:54:42.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.884+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:54:42.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.884+0000 7f7f51206ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:42.888 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.885+0000 7f7f51206ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-12-09T17:54:42.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.893+0000 7f7f1f7fe700 10 RGWWatcher::handle_notify() notify_id 592705487496 cookie 94199530769840 notifier 4724 bl.length()=612 2019-12-09T17:54:42.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.894+0000 7f7f1f7fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:54:42.897 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:42.894+0000 7f7f1f7fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:43.181 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.178+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T17:54:43.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.178+0000 7f7f51206ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:54:43.204 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.178+0000 7f7f51206ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-12-09T17:54:43.205 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.192+0000 7f7f1f7fe700 10 RGWWatcher::handle_notify() notify_id 592705486900 cookie 94199530784448 notifier 4724 bl.length()=194 2019-12-09T17:54:43.205 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.192+0000 7f7f1f7fe700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-12-09T17:54:43.206 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.192+0000 7f7f1f7fe700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-12-09T17:54:43.396 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.393+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T17:54:43.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.393+0000 7f7f51206ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:54:43.397 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.393+0000 7f7f51206ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-12-09T17:54:43.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.415+0000 7f7f1f7fe700 10 RGWWatcher::handle_notify() notify_id 592705487377 cookie 94199530773152 notifier 4724 bl.length()=183 2019-12-09T17:54:43.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.415+0000 7f7f1f7fe700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-12-09T17:54:43.418 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:43.415+0000 7f7f1f7fe700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-12-09T17:54:44.354 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.345+0000 7f7f51206ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:54:44.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.345+0000 7f7f51206ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:44.355 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.345+0000 7f7f51206ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-12-09T17:54:44.358 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.355+0000 7f7f1f7fe700 10 RGWWatcher::handle_notify() notify_id 592705487497 cookie 94199530769840 notifier 4724 bl.length()=636 2019-12-09T17:54:44.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.355+0000 7f7f1f7fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-12-09T17:54:44.359 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.355+0000 7f7f1f7fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-12-09T17:54:44.703 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T17:54:44.703 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "foo.client.0", 2019-12-09T17:54:44.703 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. foo.client.0 foo", 2019-12-09T17:54:44.704 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "foo.client.0+test@test.test", 2019-12-09T17:54:44.704 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T17:54:44.704 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T17:54:44.704 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [ 2019-12-09T17:54:44.704 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T17:54:44.705 INFO:teuthology.orchestra.run.smithi174.stdout: "id": "foo.client.0:foo", 2019-12-09T17:54:44.705 INFO:teuthology.orchestra.run.smithi174.stdout: "permissions": "full-control" 2019-12-09T17:54:44.705 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T17:54:44.705 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T17:54:44.705 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [], 2019-12-09T17:54:44.706 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [ 2019-12-09T17:54:44.706 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T17:54:44.706 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "foo.client.0:foo", 2019-12-09T17:54:44.706 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "RkbXFH2UeJl+StpxZYDmupMU55Zw2uD5fCBahVbHX73XzaZE6+y9eA==" 2019-12-09T17:54:44.706 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T17:54:44.707 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T17:54:44.708 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T17:54:44.708 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T17:54:44.708 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T17:54:44.708 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T17:54:44.708 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T17:54:44.709 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T17:54:44.710 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T17:54:44.710 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T17:54:44.710 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T17:54:44.711 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T17:54:44.711 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T17:54:44.711 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T17:54:44.711 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T17:54:44.712 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T17:54:44.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.764+0000 7f7f51206ec0 20 remove_watcher() i=0 2019-12-09T17:54:44.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.764+0000 7f7f51206ec0 2 removed watcher, disabling cache 2019-12-09T17:54:44.811 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.808+0000 7f7f51206ec0 20 remove_watcher() i=1 2019-12-09T17:54:44.863 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.860+0000 7f7f51206ec0 20 remove_watcher() i=2 2019-12-09T17:54:44.904 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.901+0000 7f7f51206ec0 20 remove_watcher() i=3 2019-12-09T17:54:44.948 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.945+0000 7f7f51206ec0 20 remove_watcher() i=4 2019-12-09T17:54:44.981 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:44.978+0000 7f7f51206ec0 20 remove_watcher() i=5 2019-12-09T17:54:45.024 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.021+0000 7f7f51206ec0 20 remove_watcher() i=6 2019-12-09T17:54:45.069 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.065+0000 7f7f51206ec0 20 remove_watcher() i=7 2019-12-09T17:54:45.077 INFO:teuthology.orchestra.run.smithi174:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret kwH+lpNvn+CB+ge2NWAdSqQY0t8CvnQZjLo9PBOQVgcaiB95/OX4EQ== --email bar.client.0+test@test.test --key-type swift --access full 2019-12-09T17:54:45.241 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.237+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.263+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.266 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.263+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.269+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.272 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.269+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.274+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.277 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.274+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.294 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.291+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:45.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.732+0000 7f30abacaec0 20 RGWRados::pool_iterate: got zonegroup_info.f02e29cf-4cae-4c96-a4a9-2001a12611b9 2019-12-09T17:54:45.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.732+0000 7f30abacaec0 20 RGWRados::pool_iterate: got zone_info.7b90ce1f-2e6d-4ddb-b794-a516be24e604 2019-12-09T17:54:45.735 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.732+0000 7f30abacaec0 20 RGWRados::pool_iterate: got zone_names.default 2019-12-09T17:54:45.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.732+0000 7f30abacaec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-12-09T17:54:45.736 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.732+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.746+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:45.749 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.746+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.759 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.756+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:45.759 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.756+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.776 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.773+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:45.776 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.773+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.781+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:45.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.781+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.791 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.789+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.792 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.789+0000 7f30abacaec0 10 cannot find current period zonegroup using local zonegroup 2019-12-09T17:54:45.792 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.789+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.797+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.800 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.797+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.805+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:45.808 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.805+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.817+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=382 2019-12-09T17:54:45.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.817+0000 7f30abacaec0 20 zonegroup default 2019-12-09T17:54:45.820 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.817+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.827 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.825+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.825+0000 7f30abacaec0 10 Cannot find current period zone using local zone 2019-12-09T17:54:45.828 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.825+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.836 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.833+0000 7f30abacaec0 20 rados_obj.operate() r=-2 bl.length=0 2019-12-09T17:54:45.836 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.833+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.850 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.847+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=46 2019-12-09T17:54:45.850 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.847+0000 7f30abacaec0 20 rados->read ofs=0 len=0 2019-12-09T17:54:45.859 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.856+0000 7f30abacaec0 20 rados_obj.operate() r=0 bl.length=951 2019-12-09T17:54:45.859 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.856+0000 7f30abacaec0 20 zone default found 2019-12-09T17:54:45.859 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.856+0000 7f30abacaec0 20 started sync module instance, tier type = 2019-12-09T17:54:45.860 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:45.856+0000 7f30abacaec0 20 started zone id=7b90ce1f-2e6d-4ddb-b794-a516be24e604 (name=default) with tier type = 2019-12-09T17:54:46.242 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=0 2019-12-09T17:54:46.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=1 2019-12-09T17:54:46.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=2 2019-12-09T17:54:46.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=3 2019-12-09T17:54:46.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=4 2019-12-09T17:54:46.243 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=5 2019-12-09T17:54:46.244 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.239+0000 7f30abacaec0 20 add_watcher() i=6 2019-12-09T17:54:46.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.276+0000 7f30abacaec0 20 add_watcher() i=7 2019-12-09T17:54:46.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.276+0000 7f30abacaec0 2 all 8 watchers are set, enabling cache 2019-12-09T17:54:46.279 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.276+0000 7f30697e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-12-09T17:54:46.281 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.278+0000 7f3098cf4700 0 -- 172.21.15.174:0/2381395869 >> [v2:172.21.15.174:3301/0,v1:172.21.15.174:6790/0] conn(0x55c4c4eab430 msgr2=0x55c4c4eb1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-12-09T17:54:46.320 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:46.317+0000 7f3098cf4700 0 -- 172.21.15.174:0/2917260963 >> [v2:172.21.15.106:6800/8833,v1:172.21.15.106:6801/8833] conn(0x7f2fdc022480 msgr2=0x7f2fdc024990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-12-09T17:54:47.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.762+0000 7f2fd37fe700 20 reqs_thread_entry: start 2019-12-09T17:54:47.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.763+0000 7f2fd17fa700 20 reqs_thread_entry: start 2019-12-09T17:54:47.766 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.763+0000 7f30abacaec0 20 init_complete bucket index max shards: 0 2019-12-09T17:54:47.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.764+0000 7f2fc37fe700 20 reqs_thread_entry: start 2019-12-09T17:54:47.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.764+0000 7f30abacaec0 20 get_system_obj_state: rctx=0x7ffc49751568 obj=default.rgw.meta:users.uid:bar.client.0 state=0x55c4c4f2ce50 s->prefetch_data=0 2019-12-09T17:54:47.767 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.764+0000 7f30abacaec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-12-09T17:54:47.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.775+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-12-09T17:54:47.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.775+0000 7f30abacaec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:47.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.775+0000 7f30abacaec0 20 get_system_obj_state: rctx=0x7ffc49751578 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x55c4c4f2ebe0 s->prefetch_data=0 2019-12-09T17:54:47.778 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.775+0000 7f30abacaec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-12-09T17:54:47.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.785+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-12-09T17:54:47.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.785+0000 7f30abacaec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:54:47.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.785+0000 7f30abacaec0 20 get_system_obj_state: rctx=0x7ffc49751188 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x55c4c4f2ebe0 s->prefetch_data=0 2019-12-09T17:54:47.788 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.785+0000 7f30abacaec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-12-09T17:54:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.793+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T17:54:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.793+0000 7f30abacaec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:54:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.793+0000 7f30abacaec0 20 get_system_obj_state: rctx=0x7ffc49751308 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x55c4c4f2ee90 s->prefetch_data=0 2019-12-09T17:54:47.796 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.793+0000 7f30abacaec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-12-09T17:54:47.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.802+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-12-09T17:54:47.805 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.802+0000 7f30abacaec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:54:47.901 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.898+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:54:47.902 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.898+0000 7f30abacaec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:47.902 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.898+0000 7f30abacaec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-12-09T17:54:47.921 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.918+0000 7f308e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487378 cookie 94303605602192 notifier 4733 bl.length()=612 2019-12-09T17:54:47.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.918+0000 7f308e7fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:54:47.922 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:47.918+0000 7f308e7fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:48.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.700+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T17:54:48.703 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.700+0000 7f30abacaec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:54:48.704 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.700+0000 7f30abacaec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-12-09T17:54:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.715+0000 7f308e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487379 cookie 94303605602192 notifier 4733 bl.length()=194 2019-12-09T17:54:48.718 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.715+0000 7f308e7fc700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-12-09T17:54:48.719 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:48.715+0000 7f308e7fc700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-12-09T17:54:49.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.112+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T17:54:49.115 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.112+0000 7f30abacaec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:54:49.116 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.112+0000 7f30abacaec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-12-09T17:54:49.131 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.128+0000 7f308e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487380 cookie 94303605611648 notifier 4733 bl.length()=183 2019-12-09T17:54:49.132 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.128+0000 7f308e7fc700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-12-09T17:54:49.132 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.128+0000 7f308e7fc700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-12-09T17:54:49.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.765+0000 7f30abacaec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:54:49.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.765+0000 7f30abacaec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:49.768 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.765+0000 7f30abacaec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-12-09T17:54:49.783 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.780+0000 7f308e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487381 cookie 94303605602192 notifier 4733 bl.length()=636 2019-12-09T17:54:49.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.780+0000 7f308e7fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-12-09T17:54:49.784 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.780+0000 7f308e7fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-12-09T17:54:49.898 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2019-12-09T17:54:49.898 INFO:teuthology.orchestra.run.smithi174.stdout: "user_id": "bar.client.0", 2019-12-09T17:54:49.898 INFO:teuthology.orchestra.run.smithi174.stdout: "display_name": "Mr. bar.client.0 bar", 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "email": "bar.client.0+test@test.test", 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "suspended": 0, 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "max_buckets": 1000, 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "subusers": [ 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "id": "bar.client.0:bar", 2019-12-09T17:54:49.899 INFO:teuthology.orchestra.run.smithi174.stdout: "permissions": "full-control" 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: "keys": [], 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: "swift_keys": [ 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: { 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: "user": "bar.client.0:bar", 2019-12-09T17:54:49.900 INFO:teuthology.orchestra.run.smithi174.stdout: "secret_key": "kwH+lpNvn+CB+ge2NWAdSqQY0t8CvnQZjLo9PBOQVgcaiB95/OX4EQ==" 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: } 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: ], 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": [], 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: "op_mask": "read, write, delete", 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: "default_placement": "", 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: "default_storage_class": "", 2019-12-09T17:54:49.901 INFO:teuthology.orchestra.run.smithi174.stdout: "placement_tags": [], 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "bucket_quota": { 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T17:54:49.902 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "user_quota": { 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "enabled": false, 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "check_on_raw": false, 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size": -1, 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "max_size_kb": 0, 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: "max_objects": -1 2019-12-09T17:54:49.903 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2019-12-09T17:54:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: "temp_url_keys": [], 2019-12-09T17:54:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: "type": "rgw", 2019-12-09T17:54:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: "mfa_ids": [] 2019-12-09T17:54:49.904 INFO:teuthology.orchestra.run.smithi174.stdout:} 2019-12-09T17:54:49.904 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T17:54:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.940+0000 7f30abacaec0 20 remove_watcher() i=0 2019-12-09T17:54:49.944 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.940+0000 7f30abacaec0 2 removed watcher, disabling cache 2019-12-09T17:54:49.985 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:49.982+0000 7f30abacaec0 20 remove_watcher() i=1 2019-12-09T17:54:50.027 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.024+0000 7f30abacaec0 20 remove_watcher() i=2 2019-12-09T17:54:50.064 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.061+0000 7f30abacaec0 20 remove_watcher() i=3 2019-12-09T17:54:50.100 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.097+0000 7f30abacaec0 20 remove_watcher() i=4 2019-12-09T17:54:50.145 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.130+0000 7f30abacaec0 20 remove_watcher() i=5 2019-12-09T17:54:50.165 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.161+0000 7f30abacaec0 20 remove_watcher() i=6 2019-12-09T17:54:50.198 INFO:teuthology.orchestra.run.smithi174.stderr:2019-12-09T17:54:50.195+0000 7f30abacaec0 20 remove_watcher() i=7 2019-12-09T17:54:50.204 INFO:tasks.swift:Configuring testswift... 2019-12-09T17:54:50.204 INFO:teuthology.orchestra.run.smithi174:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-12-09T17:54:52.735 INFO:teuthology.orchestra.run.smithi174.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-12-09T17:54:53.963 INFO:teuthology.orchestra.run.smithi174.stdout:Installing setuptools, pip, wheel...done. 2019-12-09T17:54:54.299 INFO:teuthology.orchestra.run.smithi174.stderr:Cache entry deserialization failed, entry ignored 2019-12-09T17:54:54.423 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting pip 2019-12-09T17:54:54.505 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-12-09T17:54:54.571 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: pip 2019-12-09T17:54:54.572 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: pip 9.0.1 2019-12-09T17:54:54.633 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling pip-9.0.1: 2019-12-09T17:54:54.680 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled pip-9.0.1 2019-12-09T17:54:55.298 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed pip-19.3.1 2019-12-09T17:54:55.576 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T17:54:56.104 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting setuptools==32.3.1 2019-12-09T17:54:56.110 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-12-09T17:54:56.159 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: setuptools 2019-12-09T17:54:56.159 INFO:teuthology.orchestra.run.smithi174.stdout: Found existing installation: setuptools 28.8.0 2019-12-09T17:54:56.174 INFO:teuthology.orchestra.run.smithi174.stdout: Uninstalling setuptools-28.8.0: 2019-12-09T17:54:56.183 INFO:teuthology.orchestra.run.smithi174.stdout: Successfully uninstalled setuptools-28.8.0 2019-12-09T17:54:56.324 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed setuptools-32.3.1 2019-12-09T17:54:56.879 INFO:teuthology.orchestra.run.smithi174.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-12-09T17:54:57.143 INFO:teuthology.orchestra.run.smithi174.stdout:Processing /home/ubuntu/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67/PyYAML-5.2-cp27-cp27mu-linux_x86_64.whl 2019-12-09T17:54:57.211 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting nose>=1.0.0 2019-12-09T17:54:57.213 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-12-09T17:54:57.333 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting boto>=2.0b4 2019-12-09T17:54:57.339 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-12-09T17:54:57.478 INFO:teuthology.orchestra.run.smithi174.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-12-09T17:54:57.754 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting gevent>=0.14 2019-12-09T17:54:57.770 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:54:58.022 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting isodate>=0.4.4 2019-12-09T17:54:58.027 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-12-09T17:54:58.103 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting eventlet>=0.9.14 2019-12-09T17:54:58.216 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-12-09T17:54:58.445 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting netifaces>=0.5 2019-12-09T17:54:58.447 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:54:58.525 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting xattr<0.7.0,>=0.6 2019-12-09T17:54:58.534 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-12-09T17:54:58.921 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting simplejson>=2.1.3 2019-12-09T17:54:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-12-09T17:54:59.289 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting WebOb==1.0 2019-12-09T17:54:59.300 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-12-09T17:54:59.721 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-12-09T17:54:59.722 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-12-09T17:54:59.789 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting six 2019-12-09T17:54:59.791 INFO:teuthology.orchestra.run.smithi174.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-12-09T17:54:59.843 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting monotonic>=1.4 2019-12-09T17:54:59.853 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-12-09T17:54:59.940 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting dnspython>=1.15.0 2019-12-09T17:54:59.953 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-12-09T17:55:00.058 INFO:teuthology.orchestra.run.smithi174.stdout:Collecting enum34; python_version < "3.4" 2019-12-09T17:55:00.068 INFO:teuthology.orchestra.run.smithi174.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-12-09T17:55:00.092 INFO:teuthology.orchestra.run.smithi174.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-12-09T17:55:00.093 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for xattr (setup.py): started 2019-12-09T17:55:00.579 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-12-09T17:55:00.580 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24123 sha256=ca396aeb24753fa1d7e8d7f4923d92ef695ec0ea9bb5ff55d02b87c8b5c7315c 2019-12-09T17:55:00.580 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-12-09T17:55:00.589 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for simplejson (setup.py): started 2019-12-09T17:55:01.674 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-12-09T17:55:01.675 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113128 sha256=bdb1400c6e4aad980997e8c57a14f2ba91c9272554d0928b5c9cc0bac38762e2 2019-12-09T17:55:01.676 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-12-09T17:55:01.696 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for WebOb (setup.py): started 2019-12-09T17:55:01.981 INFO:teuthology.orchestra.run.smithi174.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-12-09T17:55:01.982 INFO:teuthology.orchestra.run.smithi174.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=51559be434643a5df50e5a1e9002935b2e96d7820e4aac5c8a0f8df812b5054e 2019-12-09T17:55:01.982 INFO:teuthology.orchestra.run.smithi174.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-12-09T17:55:01.989 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully built xattr simplejson WebOb 2019-12-09T17:55:02.034 INFO:teuthology.orchestra.run.smithi174.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-12-09T17:55:03.549 INFO:teuthology.orchestra.run.smithi174.stdout:Successfully installed PyYAML-5.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-12-09T17:55:03.824 INFO:teuthology.orchestra.run.smithi174.stdout:running develop 2019-12-09T17:55:03.870 INFO:teuthology.orchestra.run.smithi174.stdout:running egg_info 2019-12-09T17:55:03.871 INFO:teuthology.orchestra.run.smithi174.stdout:creating swift.egg-info 2019-12-09T17:55:03.871 INFO:teuthology.orchestra.run.smithi174.stdout:writing swift.egg-info/PKG-INFO 2019-12-09T17:55:03.871 INFO:teuthology.orchestra.run.smithi174.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-12-09T17:55:03.871 INFO:teuthology.orchestra.run.smithi174.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-12-09T17:55:03.875 INFO:teuthology.orchestra.run.smithi174.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-12-09T17:55:03.875 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:55:03.878 INFO:teuthology.orchestra.run.smithi174.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:55:03.879 INFO:teuthology.orchestra.run.smithi174.stdout:reading manifest template 'MANIFEST.in' 2019-12-09T17:55:03.879 INFO:teuthology.orchestra.run.smithi174.stderr:warning: no files found matching 'ChangeLog' 2019-12-09T17:55:03.882 INFO:teuthology.orchestra.run.smithi174.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-12-09T17:55:03.884 INFO:teuthology.orchestra.run.smithi174.stdout:running build_ext 2019-12-09T17:55:03.884 INFO:teuthology.orchestra.run.smithi174.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-12-09T17:55:03.884 INFO:teuthology.orchestra.run.smithi174.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-12-09T17:55:03.900 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.901 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.904 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.905 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.906 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.906 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.907 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.908 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.909 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.910 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.911 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.912 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.913 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.914 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.915 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.916 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.917 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.919 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.919 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.920 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.921 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.925 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.927 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.936 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.938 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.941 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.943 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.947 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.949 INFO:teuthology.orchestra.run.smithi174.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-12-09T17:55:03.949 INFO:teuthology.orchestra.run.smithi174.stdout: 2019-12-09T17:55:03.949 INFO:teuthology.orchestra.run.smithi174.stdout:Installed /home/ubuntu/cephtest/swift 2019-12-09T17:55:03.950 INFO:teuthology.orchestra.run.smithi174.stdout:Processing dependencies for swift==1.4.4 2019-12-09T17:55:03.951 INFO:teuthology.orchestra.run.smithi174.stdout:Finished processing dependencies for swift==1.4.4 2019-12-09T17:55:03.966 INFO:teuthology.orchestra.run.smithi174:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-12-09T17:55:04.033 INFO:teuthology.orchestra.run.smithi174:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-12-09T17:55:21.311 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:34:13.161 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.317 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:34:13.167 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.348 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:34:13.173 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.380 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:34:13.205 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.391 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:34:13.214 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.397 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:34:13.218 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.446 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:34:13.297 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.508 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:34:13.333 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:21.528 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:34:13.353 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:55:37.053 INFO:teuthology.orchestra.run.smithi174.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-12-09T17:55:50.482 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-12-09T17:56:39.072 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:35:30.923 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.086 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:35:30.911 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.088 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:35:30.913 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.091 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:35:30.916 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.091 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:35:30.942 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.093 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:35:30.918 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.150 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:35:30.975 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.180 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:35:31.005 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T17:56:39.201 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:35:31.052 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:00:47.727 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-12-09T18:01:01.188 INFO:teuthology.orchestra.run.smithi174.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-12-09T18:01:02.397 INFO:teuthology.orchestra.run.smithi174.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-12-09T18:01:18.456 INFO:teuthology.orchestra.run.smithi174.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-12-09T18:01:36.460 INFO:teuthology.orchestra.run.smithi174.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-12-09T18:01:49.501 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:40:41.352 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.517 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:40:41.362 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.532 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:40:41.357 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.535 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:40:41.360 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.536 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:40:41.361 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.540 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:40:41.365 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.721 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:40:41.546 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.751 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:40:41.576 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:01:49.773 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:40:41.624 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:02:30.215 INFO:teuthology.orchestra.run.smithi174.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-12-09T18:02:30.734 INFO:teuthology.orchestra.run.smithi174.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-12-09T18:03:43.067 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:42:34.917 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.072 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:42:34.922 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.088 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:42:34.913 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.091 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:42:34.916 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.134 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:42:34.959 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.138 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:42:34.963 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.159 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:42:35.009 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.183 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:42:35.007 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:03:43.203 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:42:35.027 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.069 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:45:51.919 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.074 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:45:51.925 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.087 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:45:51.911 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.092 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:45:51.917 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.099 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:45:51.923 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.104 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:45:51.929 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.258 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:45:52.108 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.270 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:45:52.095 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:07:00.275 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:45:52.100 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:08:43.316 INFO:teuthology.orchestra.run.smithi174.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-12-09T18:08:47.016 INFO:teuthology.orchestra.run.smithi174.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-12-09T18:09:19.311 INFO:teuthology.orchestra.run.smithi174.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:09:31.314 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:12:04.991 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:50:56.841 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.003 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:50:56.853 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.048 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:50:56.873 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.064 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:50:56.888 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.072 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:50:56.896 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.075 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:50:56.899 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.169 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:50:57.019 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.204 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:50:57.029 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:05.236 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:50:57.061 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.412 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:51:02.262 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.431 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:51:02.282 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.433 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:51:02.258 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.437 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:51:02.262 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.438 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:51:02.263 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.453 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:51:02.278 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.479 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:51:02.304 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.480 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:51:02.330 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:12:10.482 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:51:02.307 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:13:55.095 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:14:06.671 INFO:teuthology.orchestra.run.smithi174.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:14:07.697 INFO:teuthology.orchestra.run.smithi174.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:14:20.194 INFO:teuthology.orchestra.run.smithi174.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:14:35.859 INFO:teuthology.orchestra.run.smithi174.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:16:02.189 INFO:teuthology.orchestra.run.smithi174.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:16:02.378 INFO:teuthology.orchestra.run.smithi174.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-12-09T18:17:22.264 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:56:14.114 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.268 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:56:14.118 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.286 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:56:14.111 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.286 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:56:14.111 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.289 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:56:14.114 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.290 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:56:14.115 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.461 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:56:14.311 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.469 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:56:14.293 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:17:22.523 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:56:14.348 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.869 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:59:18.720 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.928 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:59:18.778 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.969 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:59:18.793 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.969 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:59:18.794 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.970 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:59:18.794 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:26.990 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:59:18.815 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:27.043 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:06:59:18.868 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:27.069 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:06:59:18.894 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:20:27.076 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:06:59:18.926 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:24.437 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:27.340 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:29.394 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:32.459 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:32.490 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:01:24.340 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.497 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:01:24.347 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.503 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:01:24.328 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.507 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:01:24.332 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.508 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:01:24.333 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.512 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:01:24.337 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.580 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:01:24.430 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.601 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:01:24.426 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.621 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:01:24.446 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:22:32.962 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:33.612 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:42.651 INFO:teuthology.orchestra.run.smithi174.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:49.693 INFO:teuthology.orchestra.run.smithi174.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-12-09T18:22:59.952 INFO:teuthology.orchestra.run.smithi174.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T18:23:00.570 INFO:teuthology.orchestra.run.smithi174.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T18:23:08.233 INFO:teuthology.orchestra.run.smithi174.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-12-09T18:23:08.489 INFO:teuthology.orchestra.run.smithi174.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-12-09T18:24:46.525 INFO:teuthology.orchestra.run.smithi174.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-12-09T18:24:50.781 INFO:teuthology.orchestra.run.smithi174.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-12-09T18:25:04.723 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:04.701+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:05.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:05.723+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:06.047 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:25:06.038+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.175314+0000 front 2019-12-09T18:24:22.925173+0000 (oldest deadline 2019-12-09T18:25:05.932207+0000) 2019-12-09T18:25:06.279 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:25:06.270+0000 1694f700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.322907+0000 front 2019-12-09T18:24:22.175583+0000 (oldest deadline 2019-12-09T18:25:06.171601+0000) 2019-12-09T18:25:06.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:25:06.369+0000 1694f700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:20.796019+0000 front 2019-12-09T18:24:20.772979+0000 (oldest deadline 2019-12-09T18:25:05.375895+0000) 2019-12-09T18:25:06.714 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:06.709+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:07.065 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:25:07.060+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.175314+0000 front 2019-12-09T18:24:22.925173+0000 (oldest deadline 2019-12-09T18:25:05.932207+0000) 2019-12-09T18:25:07.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:25:07.307+0000 1694f700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.322907+0000 front 2019-12-09T18:24:22.175583+0000 (oldest deadline 2019-12-09T18:25:06.171601+0000) 2019-12-09T18:25:07.388 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:25:07.377+0000 1694f700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:20.796019+0000 front 2019-12-09T18:24:20.772979+0000 (oldest deadline 2019-12-09T18:25:05.375895+0000) 2019-12-09T18:25:07.537 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:25:07.528+0000 1694f700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.818513+0000 front 2019-12-09T18:24:22.400394+0000 (oldest deadline 2019-12-09T18:25:07.448797+0000) 2019-12-09T18:25:07.677 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:07.672+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:07.982 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:25:07.974+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:23.211618+0000 front 2019-12-09T18:24:23.218191+0000 (oldest deadline 2019-12-09T18:25:07.850233+0000) 2019-12-09T18:25:08.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:25:08.027+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.175314+0000 front 2019-12-09T18:24:22.925173+0000 (oldest deadline 2019-12-09T18:25:05.932207+0000) 2019-12-09T18:25:08.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:25:08.307+0000 1694f700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.322907+0000 front 2019-12-09T18:24:22.175583+0000 (oldest deadline 2019-12-09T18:25:06.171601+0000) 2019-12-09T18:25:08.420 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:25:08.416+0000 1694f700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:20.796019+0000 front 2019-12-09T18:24:20.772979+0000 (oldest deadline 2019-12-09T18:25:05.375895+0000) 2019-12-09T18:25:08.556 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:25:08.551+0000 1694f700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.818513+0000 front 2019-12-09T18:24:22.400394+0000 (oldest deadline 2019-12-09T18:25:07.448797+0000) 2019-12-09T18:25:08.683 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:08.678+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:08.946 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:25:08.940+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:23.211618+0000 front 2019-12-09T18:24:23.218191+0000 (oldest deadline 2019-12-09T18:25:07.850233+0000) 2019-12-09T18:25:09.049 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:25:09.038+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.175314+0000 front 2019-12-09T18:24:22.925173+0000 (oldest deadline 2019-12-09T18:25:05.932207+0000) 2019-12-09T18:25:09.375 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:25:09.361+0000 1694f700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.322907+0000 front 2019-12-09T18:24:22.175583+0000 (oldest deadline 2019-12-09T18:25:06.171601+0000) 2019-12-09T18:25:09.446 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:25:09.433+0000 1694f700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:20.796019+0000 front 2019-12-09T18:24:20.772979+0000 (oldest deadline 2019-12-09T18:25:05.375895+0000) 2019-12-09T18:25:09.542 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:25:09.537+0000 1694f700 -1 osd.7 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.818513+0000 front 2019-12-09T18:24:22.400394+0000 (oldest deadline 2019-12-09T18:25:07.448797+0000) 2019-12-09T18:25:09.674 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:25:09.670+0000 1694f700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.154388+0000 front 2019-12-09T18:24:21.264676+0000 (oldest deadline 2019-12-09T18:25:03.748593+0000) 2019-12-09T18:25:09.928 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:25:09.921+0000 1694f700 -1 osd.3 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:23.211618+0000 front 2019-12-09T18:24:23.218191+0000 (oldest deadline 2019-12-09T18:25:07.850233+0000) 2019-12-09T18:25:10.019 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:25:10.015+0000 1694f700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:22.175314+0000 front 2019-12-09T18:24:22.925173+0000 (oldest deadline 2019-12-09T18:25:05.932207+0000) 2019-12-09T18:25:10.476 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:25:10.473+0000 1694f700 -1 osd.5 138 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:24:20.796019+0000 front 2019-12-09T18:24:20.772979+0000 (oldest deadline 2019-12-09T18:25:05.375895+0000) 2019-12-09T18:25:36.539 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:25:36.430+0000 36790700 -1 osd.4 143 build_incremental_map_msg missing incremental map 143 2019-12-09T18:25:36.635 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:25:36.443+0000 124b0700 -1 osd.4 143 build_incremental_map_msg missing incremental map 143 2019-12-09T18:25:36.685 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:25:36.497+0000 3578e700 -1 osd.4 143 build_incremental_map_msg missing incremental map 143 2019-12-09T18:26:07.194 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:26:07.188+0000 1694f700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:26.872498+0000 front 2019-12-09T18:25:24.440999+0000 (oldest deadline 2019-12-09T18:26:06.775155+0000) 2019-12-09T18:26:08.203 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:26:08.199+0000 1694f700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:26.872498+0000 front 2019-12-09T18:25:24.440999+0000 (oldest deadline 2019-12-09T18:26:06.775155+0000) 2019-12-09T18:26:08.683 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:26:08.676+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:22.484686+0000 front 2019-12-09T18:25:22.660890+0000 (oldest deadline 2019-12-09T18:26:08.381253+0000) 2019-12-09T18:26:08.876 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:26:08.871+0000 1694f700 -1 osd.2 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:22.441487+0000 front 2019-12-09T18:25:22.015894+0000 (oldest deadline 2019-12-09T18:26:07.890427+0000) 2019-12-09T18:26:09.246 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:26:09.242+0000 1694f700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:26.872498+0000 front 2019-12-09T18:25:24.440999+0000 (oldest deadline 2019-12-09T18:26:06.775155+0000) 2019-12-09T18:26:09.563 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:26:09.555+0000 1694f700 -1 osd.6 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:23.522093+0000 front 2019-12-09T18:25:23.520640+0000 (oldest deadline 2019-12-09T18:26:08.799778+0000) 2019-12-09T18:26:09.719 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:26:09.714+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:22.484686+0000 front 2019-12-09T18:25:22.660890+0000 (oldest deadline 2019-12-09T18:26:08.381253+0000) 2019-12-09T18:26:09.829 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:26:09.822+0000 1694f700 -1 osd.4 144 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e143) 2019-12-09T18:26:09.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:26:09.896+0000 1694f700 -1 osd.2 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:22.441487+0000 front 2019-12-09T18:25:22.015894+0000 (oldest deadline 2019-12-09T18:26:07.890427+0000) 2019-12-09T18:26:10.111 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:26:10.106+0000 1694f700 -1 osd.7 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:26.501496+0000 front 2019-12-09T18:25:26.489837+0000 (oldest deadline 2019-12-09T18:26:09.371973+0000) 2019-12-09T18:26:10.226 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:26:10.221+0000 1694f700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:26.872498+0000 front 2019-12-09T18:25:24.440999+0000 (oldest deadline 2019-12-09T18:26:06.775155+0000) 2019-12-09T18:26:10.685 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:26:10.680+0000 1694f700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:25:22.484686+0000 front 2019-12-09T18:25:22.660890+0000 (oldest deadline 2019-12-09T18:26:08.381253+0000) 2019-12-09T18:26:11.051 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:26:11.047+0000 1694f700 -1 osd.4 145 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e143) 2019-12-09T18:26:13.383 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:05.234 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.400 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:05.251 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.456 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:05.280 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.477 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:05.302 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.486 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:05.311 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.500 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:05.325 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.644 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:05.494 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.682 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:05.507 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:26:13.727 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:05.552 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.391 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:54.241 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.398 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:54.247 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.403 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:54.228 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.408 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:54.232 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.408 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:54.233 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.410 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:54.235 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.462 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:05:54.312 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.467 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:05:54.292 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:27:02.480 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:05:54.295 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:23.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:23.540+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:24.523 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:24.517+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:25.318 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:28:25.313+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:45.265024+0000 front 2019-12-09T18:27:43.554013+0000 (oldest deadline 2019-12-09T18:28:25.027251+0000) 2019-12-09T18:28:25.409 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:28:25.390+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.170889+0000 front 2019-12-09T18:27:44.478038+0000 (oldest deadline 2019-12-09T18:28:25.180354+0000) 2019-12-09T18:28:25.459 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:28:25.455+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.480652+0000 front 2019-12-09T18:27:43.409343+0000 (oldest deadline 2019-12-09T18:28:24.776269+0000) 2019-12-09T18:28:25.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:25.561+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:25.647 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:28:25.642+0000 1694f700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:41.886371+0000 front 2019-12-09T18:27:44.919533+0000 (oldest deadline 2019-12-09T18:28:24.851354+0000) 2019-12-09T18:28:26.351 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:28:26.346+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:45.265024+0000 front 2019-12-09T18:27:43.554013+0000 (oldest deadline 2019-12-09T18:28:25.027251+0000) 2019-12-09T18:28:26.443 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:28:26.439+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.170889+0000 front 2019-12-09T18:27:44.478038+0000 (oldest deadline 2019-12-09T18:28:25.180354+0000) 2019-12-09T18:28:26.451 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:28:26.446+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.551655+0000 front 2019-12-09T18:27:45.245935+0000 (oldest deadline 2019-12-09T18:28:26.185292+0000) 2019-12-09T18:28:26.489 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:28:26.484+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.480652+0000 front 2019-12-09T18:27:43.409343+0000 (oldest deadline 2019-12-09T18:28:24.776269+0000) 2019-12-09T18:28:26.580 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:26.576+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:26.603 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:28:26.597+0000 1694f700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:41.886371+0000 front 2019-12-09T18:27:44.919533+0000 (oldest deadline 2019-12-09T18:28:24.851354+0000) 2019-12-09T18:28:26.973 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:28:26.968+0000 1694f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.955573+0000 front 2019-12-09T18:27:44.639917+0000 (oldest deadline 2019-12-09T18:28:26.752913+0000) 2019-12-09T18:28:27.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:28:27.309+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:45.265024+0000 front 2019-12-09T18:27:43.554013+0000 (oldest deadline 2019-12-09T18:28:25.027251+0000) 2019-12-09T18:28:27.409 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:28:27.405+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.551655+0000 front 2019-12-09T18:27:45.245935+0000 (oldest deadline 2019-12-09T18:28:26.185292+0000) 2019-12-09T18:28:27.451 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:28:27.447+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.480652+0000 front 2019-12-09T18:27:43.409343+0000 (oldest deadline 2019-12-09T18:28:24.776269+0000) 2019-12-09T18:28:27.484 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:28:27.480+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.170889+0000 front 2019-12-09T18:27:44.478038+0000 (oldest deadline 2019-12-09T18:28:25.180354+0000) 2019-12-09T18:28:27.558 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:27.550+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:27.655 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:28:27.650+0000 1694f700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:41.886371+0000 front 2019-12-09T18:27:44.919533+0000 (oldest deadline 2019-12-09T18:28:24.851354+0000) 2019-12-09T18:28:28.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:28:28.011+0000 1694f700 -1 osd.7 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.955573+0000 front 2019-12-09T18:27:44.639917+0000 (oldest deadline 2019-12-09T18:28:26.752913+0000) 2019-12-09T18:28:28.301 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:28:28.296+0000 1694f700 -1 osd.6 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:45.265024+0000 front 2019-12-09T18:27:43.554013+0000 (oldest deadline 2019-12-09T18:28:25.027251+0000) 2019-12-09T18:28:28.432 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:28:28.428+0000 1694f700 -1 osd.2 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.551655+0000 front 2019-12-09T18:27:45.245935+0000 (oldest deadline 2019-12-09T18:28:26.185292+0000) 2019-12-09T18:28:28.440 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:28:28.435+0000 1694f700 -1 osd.3 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.170889+0000 front 2019-12-09T18:27:44.478038+0000 (oldest deadline 2019-12-09T18:28:25.180354+0000) 2019-12-09T18:28:28.447 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:28:28.442+0000 1694f700 -1 osd.0 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:44.480652+0000 front 2019-12-09T18:27:43.409343+0000 (oldest deadline 2019-12-09T18:28:24.776269+0000) 2019-12-09T18:28:28.619 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:28:28.614+0000 1694f700 -1 osd.1 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:43.570117+0000 front 2019-12-09T18:27:40.127099+0000 (oldest deadline 2019-12-09T18:28:23.331304+0000) 2019-12-09T18:28:28.698 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:28:28.694+0000 1694f700 -1 osd.5 150 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:27:41.886371+0000 front 2019-12-09T18:27:44.919533+0000 (oldest deadline 2019-12-09T18:28:24.851354+0000) 2019-12-09T18:28:49.125 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:07:40.975 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.150 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:07:41.000 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.157 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:07:40.978 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.158 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:07:40.980 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.162 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:07:40.987 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.193 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:07:41.018 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.238 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:07:41.081 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.249 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:07:41.074 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:28:49.321 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:07:41.146 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:29:25.731 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:25.726+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:26.294 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:26.289+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:26.729 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:26.725+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:27.252 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:27.248+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:27.700 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:27.695+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:28.247 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:28.240+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:28.724 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:28.719+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:28.724 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:29:28.720+0000 1694f700 -1 osd.7 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.367938+0000 front 2019-12-09T18:28:43.024965+0000 (oldest deadline 2019-12-09T18:29:28.127254+0000) 2019-12-09T18:29:29.287 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:29.283+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:29.425 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:29:29.420+0000 1694f700 -1 osd.0 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.375609+0000 front 2019-12-09T18:28:43.371468+0000 (oldest deadline 2019-12-09T18:29:28.643546+0000) 2019-12-09T18:29:29.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:29.725+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:29.761 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:29:29.748+0000 1694f700 -1 osd.7 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.367938+0000 front 2019-12-09T18:28:43.024965+0000 (oldest deadline 2019-12-09T18:29:28.127254+0000) 2019-12-09T18:29:30.077 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:29:30.071+0000 1694f700 -1 osd.6 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.001604+0000 front 2019-12-09T18:28:45.199882+0000 (oldest deadline 2019-12-09T18:29:29.621613+0000) 2019-12-09T18:29:30.321 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:30.318+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:30.460 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:29:30.455+0000 1694f700 -1 osd.0 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.375609+0000 front 2019-12-09T18:28:43.371468+0000 (oldest deadline 2019-12-09T18:29:28.643546+0000) 2019-12-09T18:29:30.760 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:30.755+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:30.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:29:30.795+0000 1694f700 -1 osd.7 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.367938+0000 front 2019-12-09T18:28:43.024965+0000 (oldest deadline 2019-12-09T18:29:28.127254+0000) 2019-12-09T18:29:30.820 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:29:30.815+0000 1694f700 -1 osd.1 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:46.107665+0000 front 2019-12-09T18:28:45.170642+0000 (oldest deadline 2019-12-09T18:29:30.156631+0000) 2019-12-09T18:29:31.085 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:29:31.080+0000 1694f700 -1 osd.6 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.001604+0000 front 2019-12-09T18:28:45.199882+0000 (oldest deadline 2019-12-09T18:29:29.621613+0000) 2019-12-09T18:29:31.290 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:31.286+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:31.441 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:29:31.436+0000 1694f700 -1 osd.0 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.375609+0000 front 2019-12-09T18:28:43.371468+0000 (oldest deadline 2019-12-09T18:29:28.643546+0000) 2019-12-09T18:29:31.735 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:31.730+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:31.814 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:29:31.809+0000 1694f700 -1 osd.1 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:46.107665+0000 front 2019-12-09T18:28:45.170642+0000 (oldest deadline 2019-12-09T18:29:30.156631+0000) 2019-12-09T18:29:31.845 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:29:31.841+0000 1694f700 -1 osd.7 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.367938+0000 front 2019-12-09T18:28:43.024965+0000 (oldest deadline 2019-12-09T18:29:28.127254+0000) 2019-12-09T18:29:32.127 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:29:32.119+0000 1694f700 -1 osd.6 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.001604+0000 front 2019-12-09T18:28:45.199882+0000 (oldest deadline 2019-12-09T18:29:29.621613+0000) 2019-12-09T18:29:32.322 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:29:32.317+0000 1694f700 -1 osd.2 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:45.961059+0000 front 2019-12-09T18:28:39.780706+0000 (oldest deadline 2019-12-09T18:29:25.652120+0000) 2019-12-09T18:29:32.401 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:29:32.397+0000 1694f700 -1 osd.0 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:43.375609+0000 front 2019-12-09T18:28:43.371468+0000 (oldest deadline 2019-12-09T18:29:28.643546+0000) 2019-12-09T18:29:32.714 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:29:32.710+0000 1694f700 -1 osd.5 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:42.746704+0000 front 2019-12-09T18:28:43.509772+0000 (oldest deadline 2019-12-09T18:29:25.532206+0000) 2019-12-09T18:29:32.801 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:29:32.787+0000 1694f700 -1 osd.1 153 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:28:46.107665+0000 front 2019-12-09T18:28:45.170642+0000 (oldest deadline 2019-12-09T18:29:30.156631+0000) 2019-12-09T18:30:47.262 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T18:30:47.196+0000 20e25700 -1 RGWWatcher::handle_error cookie 821507296 err (107) Transport endpoint is not connected 2019-12-09T18:31:11.471 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:31:11.308+0000 1694f700 -1 osd.4 160 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e157) 2019-12-09T18:31:12.357 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:31:12.334+0000 1694f700 -1 osd.4 160 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e157) 2019-12-09T18:31:13.655 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:31:13.529+0000 1694f700 -1 osd.4 160 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e157) 2019-12-09T18:31:14.597 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:31:14.568+0000 1694f700 -1 osd.4 160 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=6 ondisk+retry+read+known_if_redirected e157) 2019-12-09T18:31:18.592 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:10:10.438 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.593 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:10:10.444 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.602 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:10:10.427 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.608 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:10:10.432 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.608 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:10:10.433 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.610 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:10:10.435 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.751 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:10:10.601 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.759 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:10:10.584 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:31:18.765 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:10:10.590 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:32:03.155 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:03.151+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:03.360 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:03.355+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:03.449 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:03.445+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:04.204 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:04.197+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:04.385 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:04.380+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:04.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:04.456+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:05.231 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:05.228+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:05.372 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:05.365+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:05.481 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:05.475+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:06.190 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:06.185+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:06.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:06.350+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:06.385 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:06.373+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:06.494 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:06.490+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:06.642 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:06.637+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:06.662 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:06.658+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:07.188 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:07.183+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:07.229 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:07.224+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:07.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:07.343+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:07.356 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:07.352+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:07.502 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:07.495+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:07.656 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:07.652+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:07.672 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:07.667+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:08.211 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:08.206+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:08.256 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:08.247+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:08.361 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:08.356+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:08.397 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:08.391+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:08.474 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:08.470+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:08.648 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:08.642+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:08.704 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:08.699+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:09.244 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:09.239+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:09.305 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:09.300+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:09.362 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:09.358+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:09.371 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:09.367+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:09.503 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:09.499+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:09.667 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:09.663+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:09.739 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:09.734+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:10.228 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:10.223+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:10.305 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:10.300+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:10.387 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:10.382+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:10.410 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:10.405+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:10.494 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:10.489+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:10.625 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:10.619+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:10.784 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:10.780+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:11.273 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:11.268+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:11.304 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:11.301+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:11.424 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:11.419+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:11.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:11.429+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:11.492 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:11.487+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:11.665 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:11.660+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:11.758 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:11.753+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:12.293 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:12.289+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:12.293 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:12.288+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:12.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:12.392+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:12.447 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:12.442+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:12.481 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:12.477+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:12.649 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:12.645+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:12.801 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:12.795+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:13.251 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:13.246+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:13.262 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:13.258+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:13.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:13.350+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:13.475 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:13.469+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:13.491 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:13.486+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:13.702 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:13.698+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:13.777 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:13.770+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:14.247 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:14.242+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:14.251 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:14.246+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:14.398 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:14.394+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:14.459 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:14.455+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:14.519 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:14.515+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:14.717 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:14.712+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:14.827 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:14.821+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:15.212 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:15.206+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:15.260 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:15.255+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:15.443 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:15.438+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:15.460 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:15.454+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:15.497 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:15.492+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:15.706 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:15.702+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:15.812 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:15.807+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:16.230 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:16.224+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:16.236 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:16.230+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:16.472 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:16.468+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:16.508 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:16.501+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:16.523 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:16.519+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:16.733 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:16.729+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:16.832 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:16.826+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:17.196 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:17.191+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:17.259 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:17.253+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:17.438 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:17.434+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:17.484 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:17.479+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:17.530 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:17.524+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:17.763 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:17.758+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:17.852 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:17.844+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:18.152 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:18.147+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:18.230 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:18.224+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:18.489 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:18.484+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:18.490 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:18.486+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:18.581 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:18.576+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:18.726 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:18.721+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:18.869 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:18.862+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:19.147 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:19.141+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:19.250 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:19.246+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:19.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:32:19.465+0000 1694f700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.133909+0000 front 2019-12-09T18:31:23.459434+0000 (oldest deadline 2019-12-09T18:32:03.213835+0000) 2019-12-09T18:32:19.531 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:32:19.526+0000 1694f700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.464387+0000 front 2019-12-09T18:31:24.457816+0000 (oldest deadline 2019-12-09T18:32:03.068529+0000) 2019-12-09T18:32:19.537 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:19.532+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:19.762 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:19.758+0000 1694f700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.082594+0000 front 2019-12-09T18:31:20.064765+0000 (oldest deadline 2019-12-09T18:32:05.712727+0000) 2019-12-09T18:32:19.884 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:32:19.878+0000 1694f700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443362+0000 front 2019-12-09T18:31:24.407442+0000 (oldest deadline 2019-12-09T18:32:06.304360+0000) 2019-12-09T18:32:20.142 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:32:20.136+0000 1694f700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:24.443907+0000 front 2019-12-09T18:31:24.409556+0000 (oldest deadline 2019-12-09T18:32:06.722585+0000) 2019-12-09T18:32:20.218 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:32:20.211+0000 1694f700 -1 osd.3 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:26.570166+0000 front 2019-12-09T18:31:20.104881+0000 (oldest deadline 2019-12-09T18:32:02.695551+0000) 2019-12-09T18:32:20.494 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:32:20.490+0000 1694f700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:31:20.528382+0000 front 2019-12-09T18:31:20.464038+0000 (oldest deadline 2019-12-09T18:32:06.344997+0000) 2019-12-09T18:32:50.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:50.314+0000 1694f700 -1 osd.6 162 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:112531 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821612096 gen 73] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T18:32:51.351 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:32:51.347+0000 1694f700 -1 osd.6 162 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:112531 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821612096 gen 73] snapc 0=[] ondisk+write+known_if_redirected e161) 2019-12-09T18:33:17.411 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:33:17.254+0000 36790700 -1 osd.3 164 build_incremental_map_msg missing incremental map 164 2019-12-09T18:33:17.547 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:33:17.364+0000 36790700 -1 osd.3 164 build_incremental_map_msg missing incremental map 164 2019-12-09T18:33:17.578 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:33:17.437+0000 3378a700 -1 osd.3 164 build_incremental_map_msg missing incremental map 164 2019-12-09T18:33:18.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:18.322+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:18.369 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:18.366+0000 1694f700 -1 osd.6 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:19.291 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:19.288+0000 1694f700 -1 osd.1 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:19.350 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:19.345+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:19.400 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:19.394+0000 1694f700 -1 osd.6 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:20.223 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:20.219+0000 1694f700 -1 osd.2 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:20.323 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:20.317+0000 1694f700 -1 osd.1 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:20.372 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:20.368+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:20.442 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:20.437+0000 1694f700 -1 osd.6 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:21.244 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:21.221+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:21.350 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:21.348+0000 1694f700 -1 osd.5 165 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:21.455 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:21.432+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:21.465 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:21.357+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:22.200 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:22.196+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:22.345 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:22.340+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:22.450 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:22.446+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:22.501 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:22.498+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:23.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:23.132+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:23.241 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:23.237+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:23.318 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:23.314+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:23.448 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:23.445+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:23.463 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:23.458+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:23.706 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:23.702+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:24.156 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:24.149+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:24.220 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:24.215+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:24.325 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:24.320+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:24.447 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:24.443+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:24.504 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:24.499+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:24.682 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:24.678+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:25.155 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:25.149+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:25.214 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:25.210+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:25.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:25.350+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:25.487 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:25.481+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:25.493 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:25.488+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:25.711 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:25.707+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:26.144 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:26.139+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:26.241 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:26.235+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:26.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:26.359+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:26.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:26.444+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:26.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:26.467+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:26.715 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:26.710+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:27.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:27.130+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:27.222 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:27.218+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:27.327 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:27.320+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:27.457 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:27.451+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:27.478 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:27.471+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:27.698 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:27.693+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:28.123 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:28.117+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:28.187 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:28.181+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:28.333 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:28.328+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:28.441 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:28.436+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:28.512 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:28.507+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:28.687 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:28.682+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:29.108 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:29.103+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:29.154 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:29.148+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:29.307 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:29.302+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:29.441 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:29.435+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:29.543 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:29.538+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:29.653 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:29.649+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:30.126 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:30.120+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:30.140 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:30.134+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:30.329 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:30.323+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:30.462 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:30.456+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:30.536 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:30.531+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:30.664 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:30.660+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:31.133 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:31.128+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:31.143 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:31.139+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:31.287 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:31.281+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:31.440 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:31.435+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:31.581 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:31.576+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:31.660 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:31.654+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:32.148 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:32.142+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:32.171 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:32.165+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:32.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:32.256+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:32.431 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:32.425+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:32.573 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:32.568+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:32.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:32.624+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:33.184 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:33.179+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:33.199 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:33.193+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:33.221 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:33.216+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:33.428 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:33.423+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:33.615 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:33.609+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:33.621 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:33.616+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:34.166 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:34.161+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:34.167 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:34.161+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:34.222 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:34.218+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:34.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:34.397+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:33:34.606 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:33:34.600+0000 1694f700 -1 osd.6 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.334945+0000 front 2019-12-09T18:32:35.044798+0000 (oldest deadline 2019-12-09T18:33:17.909815+0000) 2019-12-09T18:33:34.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:33:34.612+0000 1694f700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.676573+0000 front 2019-12-09T18:32:37.674669+0000 (oldest deadline 2019-12-09T18:33:22.910968+0000) 2019-12-09T18:33:35.150 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:33:35.145+0000 1694f700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:35.335275+0000 front 2019-12-09T18:32:35.136506+0000 (oldest deadline 2019-12-09T18:33:19.822733+0000) 2019-12-09T18:33:35.168 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:33:35.164+0000 1694f700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:37.493503+0000 front 2019-12-09T18:32:37.120112+0000 (oldest deadline 2019-12-09T18:33:22.980297+0000) 2019-12-09T18:33:35.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:33:35.236+0000 1694f700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:32.107661+0000 front 2019-12-09T18:32:32.221889+0000 (oldest deadline 2019-12-09T18:33:17.927450+0000) 2019-12-09T18:33:35.435 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:33:35.430+0000 1694f700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:32:33.084853+0000 front 2019-12-09T18:32:33.084162+0000 (oldest deadline 2019-12-09T18:33:18.352068+0000) 2019-12-09T18:34:35.278 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:34:35.275+0000 32f89700 -1 osd.3 168 build_incremental_map_msg missing incremental map 164 2019-12-09T18:34:37.815 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:34:37.686+0000 3578e700 -1 osd.0 170 build_incremental_map_msg missing incremental map 170 2019-12-09T18:34:37.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:34:37.758+0000 32f89700 -1 osd.0 170 build_incremental_map_msg missing incremental map 170 2019-12-09T18:34:38.065 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:34:37.878+0000 37f93700 -1 osd.0 170 build_incremental_map_msg missing incremental map 170 2019-12-09T18:35:26.902 INFO:teuthology.orchestra.run.smithi174.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-12-09T18:35:33.566 INFO:teuthology.orchestra.run.smithi174.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-12-09T18:35:34.751 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:14:26.601 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.762 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:14:26.612 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.768 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:14:26.593 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.773 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:14:26.598 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.776 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:14:26.601 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.777 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:14:26.602 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:34.997 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:14:26.822 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:35.001 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:14:26.826 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:35.018 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:14:26.868 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:35:38.593 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:38.588+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:39.614 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:39.608+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:39.630 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:39.624+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:40.648 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:40.643+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:40.656 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:40.650+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:41.254 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:41.250+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:41.615 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:41.610+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:41.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:41.628+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:42.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:42.260+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:42.310 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:42.305+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:42.328 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:42.322+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:42.619 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:42.614+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:42.653 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:42.649+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:43.258 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:43.252+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:43.345 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:43.341+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:43.366 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:43.362+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:43.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:43.570+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:43.643 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:43.639+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:44.239 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:44.235+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:44.323 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:44.318+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:44.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:44.321+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:44.391 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:44.387+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:44.617 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:44.613+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:44.619 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:44.616+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:45.268 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:45.263+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:45.328 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:45.322+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:45.355 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:45.351+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:45.408 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:45.403+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:45.617 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:45.613+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:45.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:45.621+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:46.318 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:46.313+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:46.341 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:46.336+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:46.364 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:46.360+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:46.368 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:46.365+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:46.587 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:46.583+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:46.626 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:46.622+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:47.331 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:47.327+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:47.353 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:47.350+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:47.361 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:47.357+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:47.381 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:47.377+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:47.545 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:47.540+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:47.652 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:47.648+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:48.338 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:48.334+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:48.353 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:48.349+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:48.357 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:48.352+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:48.374 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:48.370+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:48.593 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:48.588+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:48.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:48.669+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:49.304 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:49.299+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:49.372 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:49.368+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:49.373 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:49.369+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:49.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:49.393+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:49.566 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:49.561+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:49.696 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:49.693+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:50.329 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:50.324+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:50.350 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:50.347+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:50.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:50.350+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:50.391 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:50.387+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:50.616 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:50.611+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:50.700 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:50.696+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:51.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:51.290+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:51.312 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:51.307+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:51.377 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:51.367+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:51.412 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:51.407+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:51.591 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:51.587+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:51.749 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:51.743+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:52.257 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:52.253+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:52.321 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:52.315+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:52.416 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:52.410+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:52.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:52.440+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:52.638 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:52.633+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:52.725 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:52.719+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:53.238 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:53.233+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:53.301 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:53.295+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:53.463 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:53.459+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:53.488 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:53.482+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:53.678 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:53.673+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:53.742 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:53.736+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:54.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:54.193+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:54.312 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:54.308+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:54.442 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:54.436+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:54.470 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:54.464+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:54.664 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:54.659+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:54.769 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:54.764+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:55.232 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:55.228+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:55.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:55.348+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:55.406 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:55.402+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:55.461 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:55.456+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:55.688 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:55.684+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:55.778 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:55.772+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:56.212 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:56.206+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:56.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:56.309+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:56.392 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:56.378+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:56.447 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:56.441+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:56.729 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:56.724+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:56.793 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:56.789+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:57.195 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:57.190+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:57.284 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:57.280+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:57.364 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:57.359+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:57.429 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:35:57.424+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:35:57.439 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:57.434+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:57.727 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:57.721+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:57.844 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:57.839+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:58.173 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:58.168+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:58.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:58.295+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:58.351 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:58.346+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:58.424 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:58.419+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:58.456 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:35:58.450+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:35:58.746 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:58.743+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:58.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:58.846+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:35:59.196 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:35:59.193+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:35:59.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:35:59.299+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:35:59.378 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:35:59.366+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:35:59.430 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:35:59.426+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:35:59.471 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:35:59.466+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:35:59.751 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:35:59.747+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:35:59.853 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:35:59.847+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:00.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:00.155+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:00.337 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:00.331+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:00.419 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:00.414+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:00.463 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:00.458+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:00.468 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:00.464+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:00.742 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:00.737+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:00.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:00.825+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:01.145 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:01.141+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:01.310 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:01.306+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:01.398 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:01.393+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:01.453 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:01.447+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:01.460 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:01.455+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:01.699 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:01.695+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:01.881 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:01.873+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:02.133 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:02.128+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:02.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:02.356+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:02.405 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:02.400+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:02.486 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:02.481+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:02.496 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:02.492+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:02.661 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:02.657+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:02.910 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:02.904+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:03.089 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:03.085+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:03.323 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:03.319+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:03.398 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:03.393+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:03.476 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:03.472+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:03.488 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:03.483+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:03.640 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:03.636+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:03.954 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:03.946+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:04.091 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:04.086+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:04.281 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:04.277+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:04.353 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:04.347+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:04.458 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:04.453+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:04.488 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:04.483+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:04.678 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:04.674+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:04.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:04.959+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:05.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:36:05.077+0000 1694f700 -1 osd.5 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.608795+0000 front 2019-12-09T18:34:57.594156+0000 (oldest deadline 2019-12-09T18:35:43.487936+0000) 2019-12-09T18:36:05.268 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:36:05.262+0000 1694f700 -1 osd.6 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.400972+0000 front 2019-12-09T18:34:57.569818+0000 (oldest deadline 2019-12-09T18:35:40.648316+0000) 2019-12-09T18:36:05.339 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:36:05.333+0000 1694f700 -1 osd.3 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.472405+0000 front 2019-12-09T18:34:55.897502+0000 (oldest deadline 2019-12-09T18:35:41.781906+0000) 2019-12-09T18:36:05.470 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:36:05.465+0000 1694f700 -1 osd.1 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:57.705944+0000 front 2019-12-09T18:34:57.728043+0000 (oldest deadline 2019-12-09T18:35:41.715519+0000) 2019-12-09T18:36:05.543 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:36:05.537+0000 1694f700 -1 osd.0 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.761489+0000 front 2019-12-09T18:34:59.292365+0000 (oldest deadline 2019-12-09T18:35:56.915204+0000) 2019-12-09T18:36:05.661 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:36:05.655+0000 1694f700 -1 osd.7 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:56.445095+0000 front 2019-12-09T18:34:56.446939+0000 (oldest deadline 2019-12-09T18:35:39.237046+0000) 2019-12-09T18:36:05.994 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:36:05.990+0000 1694f700 -1 osd.2 171 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:34:59.174956+0000 front 2019-12-09T18:34:57.161235+0000 (oldest deadline 2019-12-09T18:35:38.569743+0000) 2019-12-09T18:36:32.798 INFO:teuthology.orchestra.run.smithi174.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-12-09T18:37:03.958 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:03.953+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:04.628 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:04.622+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:04.959 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:04.951+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:05.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:05.121+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:05.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:05.575+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:05.649 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:05.644+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:05.934 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:05.930+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:05.985 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:05.980+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:06.122 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:06.116+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:06.565 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:06.558+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:06.636 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:06.630+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:06.955 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:06.948+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:06.991 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:06.984+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:07.062 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:07.057+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:07.172 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:07.167+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:07.571 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:07.566+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:07.687 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:07.682+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:07.985 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:07.975+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:07.988 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:07.983+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:08.112 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:08.106+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:08.224 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:08.219+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:08.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:08.608+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:08.663 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:08.658+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:08.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:08.980+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:08.992 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:08.986+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:09.145 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:09.139+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:09.225 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:09.218+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:09.639 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:09.634+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:09.691 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:09.685+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:09.973 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:09.967+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:10.038 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:10.033+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:10.139 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:10.133+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:10.248 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:10.243+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:10.610 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:10.605+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:10.723 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:10.718+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:10.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:10.988+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:11.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:11.052+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:11.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:11.158+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:11.287 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:11.258+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:11.458 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:16:03.308 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.467 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:16:03.317 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.489 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:16:03.306 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.534 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:16:03.358 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.535 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:16:03.360 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.537 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:16:03.362 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.575 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:16:03.425 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.642 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:16:03.466 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.659 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:11.655+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:11.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:16:03.495 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:37:11.744 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:11.739+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:11.987 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:11.981+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:12.105 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:12.100+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:12.164 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:12.159+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:12.310 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:12.305+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:12.679 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:12.674+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:12.781 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:12.776+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:13.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:13.011+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:13.107 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:13.102+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:13.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:13.115+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:13.284 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:13.279+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:13.675 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:13.671+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:13.818 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:13.814+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:14.070 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:14.064+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:14.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:14.084+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:14.120 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:14.115+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:14.252 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:14.246+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:14.691 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:14.685+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:14.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:14.785+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:15.047 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:15.042+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:15.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:15.091+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:15.114 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:15.109+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:15.291 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:15.287+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:15.741 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:15.736+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:15.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:15.806+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:16.037 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:16.032+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:16.069 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:16.064+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:16.116 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:16.111+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:16.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:16.309+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:16.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:16.738+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:16.785 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:16.780+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:17.006 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:17.001+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:17.110 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:17.105+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:17.114 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:17.109+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:17.369 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:17.363+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:17.760 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:17.754+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:17.814 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:17.809+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:18.034 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:18.029+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:18.070 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:18.065+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:18.073 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:18.068+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:18.334 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:18.329+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:18.777 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:18.771+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:18.813 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:18.808+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:18.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:18.981+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:19.072 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:19.064+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:19.073 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:19.069+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:19.080 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:19.074+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:19.289 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:19.284+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:19.811 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:19.806+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:19.836 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:19.830+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:20.038 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:20.032+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:20.068 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:20.064+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:20.115 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:20.103+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:20.123 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:20.116+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:20.250 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:20.244+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:20.831 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:20.826+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:20.875 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:20.869+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:21.068 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:21.058+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:21.071 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:21.066+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:21.086 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:21.080+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:21.118 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:21.112+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:21.249 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:21.244+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:21.872 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:21.864+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:21.886 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:21.878+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:22.050 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:22.046+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:22.051 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:22.043+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:22.087 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:22.082+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:22.090 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:22.086+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:22.281 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:22.276+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:22.885 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:22.879+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:22.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:22.895+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:23.065 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:23.060+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:23.067 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:23.061+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:23.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:23.086+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:23.104 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:23.095+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:23.104 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:23.100+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:23.315 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:23.310+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:23.916 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:23.911+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:23.917 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:23.912+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:24.061 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:24.055+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:24.063 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:24.055+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:24.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:24.058+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:24.078 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:24.071+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:24.086 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:24.080+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:24.334 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:24.329+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:24.905 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:24.899+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:24.962 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:24.955+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:25.015 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:25.011+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:25.035 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:25.031+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:25.112 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:25.096+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:25.116 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:25.112+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:25.123 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:25.118+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:25.299 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:25.294+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:25.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:25.896+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:25.958 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:25.953+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:25.984 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:25.979+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:26.040 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:26.035+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:26.089 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:26.080+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:26.091 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:26.084+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:26.138 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:26.132+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:26.328 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:26.323+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:26.872 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:26.867+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:26.958 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:26.953+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:27.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:27.028+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:27.084 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:27.078+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:27.094 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:27.089+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:27.108 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:27.103+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:27.109 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:27.104+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:27.330 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:27.325+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:27.829 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:27.825+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:27.983 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:27.979+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:28.011 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:28.006+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:28.069 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:28.065+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:28.071 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:28.065+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:28.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:28.067+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:28.082 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:28.077+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:28.295 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:28.290+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:28.835 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:28.830+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:28.952 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:28.948+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:29.050 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:29.044+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:29.066 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:29.060+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:29.067 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:29.063+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:29.092 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:29.086+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:29.136 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:29.131+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:29.286 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:29.281+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:29.801 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:29.797+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:29.921 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:29.918+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:30.069 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:30.065+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:30.081 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:30.077+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:30.103 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:30.098+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:30.115 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:30.107+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:30.115 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:30.109+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:30.300 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:30.290+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:30.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:30.773+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:30.949 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:30.942+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:31.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:31.074+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:31.097 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:31.091+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:31.118 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:31.111+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:31.119 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:31.114+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:31.125 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:31.121+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:31.285 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:31.277+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:31.789 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:31.784+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:31.918 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:31.914+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:32.095 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:32.092+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:32.108 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:32.103+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:32.109 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:32.105+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:32.125 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:32.121+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:32.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:32.172+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:32.271 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:32.265+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:32.806 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:32.803+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:32.955 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:32.948+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:33.073 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:33.068+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:33.082 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:33.072+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:33.084 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:33.074+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:33.132 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:33.128+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:33.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:33.130+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:33.242 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:33.239+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:33.791 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:33.786+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:33.913 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:33.909+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:34.046 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:34.041+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:34.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:34.045+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:34.112 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:34.107+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:34.122 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:34.117+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:34.147 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:34.139+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:34.268 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:34.263+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:34.800 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:34.796+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:34.916 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:34.910+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:35.060 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:35.054+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:35.063 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:35.059+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:35.075 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:35.070+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:35.131 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:35.126+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:35.154 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:35.150+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:35.321 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:35.316+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:35.787 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:35.782+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:35.957 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:35.953+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:36.019 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:36.012+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:36.019 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:36.015+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:36.053 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:36.047+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:36.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:36.129+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:36.196 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:36.191+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:36.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:36.315+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:36.506 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:37:36.482+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e177 v177) 2019-12-09T18:37:36.819 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:36.815+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:37.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:37.002+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:37.045 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:37.039+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:37.046 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:37.041+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:37.070 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:37.064+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:37.099 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:37.093+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:37.188 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:37.183+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:37.275 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:37.271+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:37.817 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:37.813+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:38.025 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:38.020+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:38.055 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:38.046+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:38.055 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:38.049+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:38.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:38.053+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:38.087 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:38.082+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:38.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:38.209+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:38.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:38.307+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:38.774 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:38.770+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:39.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:39.011+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:39.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:39.028+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:39.070 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:39.062+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:39.072 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:39.068+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:39.119 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:39.114+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:39.176 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:39.173+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:39.322 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:39.316+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:39.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:39.768+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:40.038 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:40.032+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:40.043 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:40.039+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:40.061 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:40.054+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:40.062 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:40.057+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:40.114 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:40.109+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:40.144 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:40.139+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:40.307 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:40.302+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:40.777 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:40.773+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:41.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:41.025+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:41.048 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:41.042+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:41.073 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:41.068+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:41.079 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:41.073+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:41.119 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:41.114+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:41.146 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:41.141+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:41.300 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:41.294+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:41.515 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:37:41.508+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e177 v177) 2019-12-09T18:37:41.753 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:41.748+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:42.006 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:42.001+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:42.032 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:42.027+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:42.036 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:42.032+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:42.084 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:42.079+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:42.133 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:42.128+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:42.152 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:42.148+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:42.257 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:42.251+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:42.806 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:42.801+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:43.045 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:43.040+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:43.053 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:43.047+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:43.054 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:43.049+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:43.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:43.096+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:43.145 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:43.140+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:43.186 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:43.183+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:43.276 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:43.270+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:43.806 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:43.802+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:44.058 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:44.052+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:44.059 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:44.054+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:44.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:44.056+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:44.134 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:44.129+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:44.188 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:44.182+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:44.212 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:44.208+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:44.245 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:44.240+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:44.787 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:44.781+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:45.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:45.011+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:45.086 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:45.081+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:45.089 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:45.086+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:45.162 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:45.157+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:45.180 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:45.175+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:45.235 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:45.230+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:45.282 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:45.277+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:45.829 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:45.823+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:45.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:45.994+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:46.143 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:46.137+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:46.144 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:46.139+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:46.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:46.159+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:46.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:46.201+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:46.233 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:46.227+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:46.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:46.320+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:46.543 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:37:46.535+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e177 v177) 2019-12-09T18:37:46.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:46.830+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:46.987 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:46.983+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:47.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:47.145+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:47.167 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:47.161+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:47.168 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:47.163+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:47.192 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:47.188+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:47.252 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:47.246+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:47.354 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:47.348+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:47.837 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:47.833+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:48.011 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:48.007+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:48.132 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:48.127+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:48.133 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:48.129+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:48.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:48.147+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:48.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:48.171+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:48.291 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:48.286+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:48.368 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:48.362+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:48.885 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:48.881+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:49.034 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:49.030+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:49.120 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:49.114+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:49.121 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:49.116+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:49.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:49.156+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:49.173 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:49.167+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:49.315 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:49.310+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:49.421 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:49.414+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:49.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:49.867+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:50.028 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:50.023+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:50.109 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:50.103+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:50.110 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:50.105+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:50.211 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:50.205+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:50.221 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:50.217+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:50.330 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:50.324+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:50.467 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:50.461+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:50.896 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:50.890+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:51.053 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:51.049+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:51.113 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:51.107+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:51.113 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:51.109+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:51.253 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:51.249+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:51.261 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:51.256+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:51.315 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:51.311+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:51.432 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:51.422+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:51.563 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:37:51.556+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e177 v177) 2019-12-09T18:37:51.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:51.897+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:52.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:52.069+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:52.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:52.070+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:52.084 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:52.079+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:52.256 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:52.251+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:52.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:52.286+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:52.298 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:52.292+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:52.442 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:52.437+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:52.927 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:52.923+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:53.036 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:53.030+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:53.037 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:53.033+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:53.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:53.090+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:53.239 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:53.235+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:53.290 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:53.285+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:53.306 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:53.302+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:53.495 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:53.490+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:53.916 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:53.912+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:54.075 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:54.069+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:54.076 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:54.071+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:54.138 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:54.134+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:54.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:54.269+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:54.298 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:54.293+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:54.315 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:54.309+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:54.513 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:54.508+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:54.968 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:54.964+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:55.104 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:55.099+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:55.104 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:55.100+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:55.126 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:55.122+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:55.292 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:55.287+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:55.293 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:55.288+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:55.296 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:55.292+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:55.542 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:55.537+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:55.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:55.933+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:56.144 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:56.137+0000 1694f700 -1 osd.4 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:36:42.987062+0000 (oldest deadline 2019-12-09T18:37:22.987062+0000) 2019-12-09T18:37:56.144 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:56.138+0000 1694f700 -1 osd.4 177 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:37:56.175 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:37:56.171+0000 1694f700 -1 osd.1 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.102260+0000 front 2019-12-09T18:36:21.102930+0000 (oldest deadline 2019-12-09T18:37:04.572226+0000) 2019-12-09T18:37:56.268 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:37:56.261+0000 1694f700 -1 osd.7 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:17.532580+0000 front 2019-12-09T18:36:17.927850+0000 (oldest deadline 2019-12-09T18:37:03.319150+0000) 2019-12-09T18:37:56.290 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:37:56.285+0000 1694f700 -1 osd.6 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.570256+0000 front 2019-12-09T18:36:22.557518+0000 (oldest deadline 2019-12-09T18:37:06.765944+0000) 2019-12-09T18:37:56.339 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:37:56.335+0000 1694f700 -1 osd.0 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.553473+0000 front 2019-12-09T18:36:21.321767+0000 (oldest deadline 2019-12-09T18:37:05.296789+0000) 2019-12-09T18:37:56.589 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:37:56.584+0000 1694f700 -1 osd.5 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:21.091092+0000 front 2019-12-09T18:36:20.740890+0000 (oldest deadline 2019-12-09T18:37:04.817696+0000) 2019-12-09T18:37:56.709 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:37:56.700+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e177 v177) 2019-12-09T18:37:56.959 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:37:56.956+0000 1694f700 -1 osd.2 177 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:36:22.691430+0000 front 2019-12-09T18:36:22.690545+0000 (oldest deadline 2019-12-09T18:37:04.918153+0000) 2019-12-09T18:37:57.292 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:37:57.152+0000 1694f700 -1 osd.4 178 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e175) 2019-12-09T18:38:26.435 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:26.260+0000 35f8f700 -1 osd.3 182 build_incremental_map_msg missing incremental map 182 2019-12-09T18:38:26.615 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:26.369+0000 31786700 -1 osd.3 182 build_incremental_map_msg missing incremental map 182 2019-12-09T18:38:26.661 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:26.489+0000 34f8d700 -1 osd.3 182 build_incremental_map_msg missing incremental map 182 2019-12-09T18:38:26.670 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:26.525+0000 38794700 -1 osd.3 182 build_incremental_map_msg missing incremental map 182 2019-12-09T18:38:29.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:29.392+0000 33f8b700 -1 osd.3 183 build_incremental_map_msg missing incremental map 182 2019-12-09T18:38:56.420 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:56.414+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:38:57.448 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:57.445+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:38:58.482 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:58.478+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:38:59.489 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:38:59.481+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:00.463 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:00.460+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:01.464 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:01.461+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:02.507 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:02.503+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:03.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:03.467+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:04.438 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:04.434+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:05.447 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:05.443+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:06.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:06.466+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:07.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:07.494+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:08.511 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:08.507+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:09.523 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:09.519+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:10.543 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:10.539+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:11.507 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:11.503+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:12.539 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:12.535+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:13.504 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:13.502+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:14.535 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:14.531+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:15.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:15.506+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:16.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:16.507+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:17.487 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:17.484+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:18.485 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:18.481+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:19.525 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:19.521+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:20.513 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:20.509+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:21.478 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:21.475+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:22.506 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:22.503+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:23.538 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:23.534+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:23.788 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:23.784+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:24.523 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:24.514+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:24.785 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:24.779+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:24.807 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:39:24.801+0000 1694f700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.466561+0000 front 2019-12-09T18:38:40.190275+0000 (oldest deadline 2019-12-09T18:39:24.421580+0000) 2019-12-09T18:39:25.490 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:25.485+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:25.491 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:25.487+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:25.813 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:39:25.809+0000 1694f700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.466561+0000 front 2019-12-09T18:38:40.190275+0000 (oldest deadline 2019-12-09T18:39:24.421580+0000) 2019-12-09T18:39:25.838 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:25.834+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:25.930 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:18:17.780 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:25.934 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:18:17.784 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:25.947 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:18:17.770 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:25.948 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:18:17.773 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:25.948 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:18:17.773 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:25.952 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:18:17.776 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:26.079 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:18:17.929 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:26.089 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:18:17.914 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:26.096 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:18:17.921 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:39:26.366 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:26.360+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:26.483 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:26.479+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:26.484 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:26.480+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:26.805 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:39:26.800+0000 1694f700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.466561+0000 front 2019-12-09T18:38:40.190275+0000 (oldest deadline 2019-12-09T18:39:24.421580+0000) 2019-12-09T18:39:26.845 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:26.842+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:27.349 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:27.344+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:27.447 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:27.442+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:27.451 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:27.447+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:27.451 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:27.447+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:27.801 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:39:27.797+0000 1694f700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.466561+0000 front 2019-12-09T18:38:40.190275+0000 (oldest deadline 2019-12-09T18:39:24.421580+0000) 2019-12-09T18:39:27.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:27.807+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:28.340 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:28.336+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:28.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:28.416+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:28.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:28.462+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:28.472 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:28.468+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:28.523 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:28.518+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:28.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:28.798+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:28.808 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:28.805+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:28.829 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:39:28.825+0000 1694f700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.466561+0000 front 2019-12-09T18:38:40.190275+0000 (oldest deadline 2019-12-09T18:39:24.421580+0000) 2019-12-09T18:39:29.296 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:29.292+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:29.428 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:29.423+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:29.429 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:29.420+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:29.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:29.425+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:29.574 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:29.569+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:29.808 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:29.803+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:29.857 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:29.852+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:30.309 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:30.302+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:30.457 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:30.453+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:30.461 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:30.455+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:30.462 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:30.457+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:30.549 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:30.544+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:30.782 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:30.777+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:30.869 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:30.864+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:31.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:31.287+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:31.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:31.416+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:31.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:31.418+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:31.495 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:31.491+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:31.524 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:31.518+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:31.777 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:31.772+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:31.911 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:31.906+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:32.281 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:32.275+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:32.445 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:32.440+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:32.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:32.441+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:32.475 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:32.471+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:32.516 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:32.510+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:32.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:32.763+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:32.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:32.917+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:33.270 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:33.265+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:33.468 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:33.464+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:33.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:33.490+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:33.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:33.492+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:33.553 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:33.547+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:33.798 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:33.793+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:33.946 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:33.941+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:34.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:34.235+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:34.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:34.456+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:34.495 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:34.490+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:34.496 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:34.491+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:34.522 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:34.518+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:34.770 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:34.766+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:35.002 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:34.993+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:35.290 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:35.285+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:35.515 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:35.506+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:35.542 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:35.536+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:35.542 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:35.538+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:35.556 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:35.552+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:35.737 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:35.731+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:35.962 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:35.956+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:36.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:36.294+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:36.474 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:36.469+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:36.500 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:36.495+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:36.503 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:36.499+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:36.587 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:36.582+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:36.785 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:36.773+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:36.994 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:36.988+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:37.311 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:37.308+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:37.441 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:37.436+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:37.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:37.545+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:37.552 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:37.548+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:37.572 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:37.567+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:37.743 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:37.735+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:37.958 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:37.953+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:38.342 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:38.339+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:38.425 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:38.421+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:38.538 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:38.532+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:38.540 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:38.534+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:38.609 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:38.603+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:38.700 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:38.697+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:38.968 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:38.962+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:39.346 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:39.341+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:39.399 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:39.395+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:39.493 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:39.488+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:39.494 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:39.490+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:39.595 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:39.589+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:39.668 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:39.664+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:39.973 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:39.968+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:40.386 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:40.382+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:40.398 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:40.394+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:40.530 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:40.525+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:40.532 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:40.528+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:40.631 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:40.625+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:40.656 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:40.651+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:40.950 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:40.944+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:41.414 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:41.410+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:41.426 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:41.422+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:41.587 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:41.578+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:41.602 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:41.581+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:41.616 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:41.611+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:41.644 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:41.639+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:42.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:41.998+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:42.457 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:42.451+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:42.472 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:42.467+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:42.601 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:42.596+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:42.603 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:42.599+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:42.604 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:42.599+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:42.639 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:42.634+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:43.024 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:43.019+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:43.483 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:43.477+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:43.517 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:43.511+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:43.576 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:43.570+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:43.576 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:43.571+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:43.644 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:43.637+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:43.647 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:43.642+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:44.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:44.056+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:44.514 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:44.509+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:44.523 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:44.518+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:44.566 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:44.560+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:44.566 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:44.562+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:44.638 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:44.633+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:44.666 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:44.661+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:45.069 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:45.064+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:45.489 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:45.484+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:45.517 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:45.513+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:45.522 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:45.517+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:45.522 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:45.518+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:45.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:45.621+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:45.670 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:45.665+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:46.060 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:46.054+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:46.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:46.477+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:46.517 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:46.513+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:46.542 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:46.537+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:46.543 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:46.538+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:46.639 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:46.634+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:46.717 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:46.713+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:47.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:47.072+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:47.506 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:47.502+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:47.508 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:47.504+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:47.572 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:47.567+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:47.574 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:47.569+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:47.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:47.638+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:47.728 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:47.723+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:48.098 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:48.093+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:48.464 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:48.460+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:48.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:48.488+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:48.600 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:48.596+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:48.621 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:48.616+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:48.623 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:48.619+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:48.727 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:48.722+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:49.083 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:49.077+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:49.448 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:49.444+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:49.491 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:49.486+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:49.578 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:49.572+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:49.579 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:49.574+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:49.626 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:49.620+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:49.747 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:49.742+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:50.075 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:50.069+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:50.423 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:50.418+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:50.489 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:50.484+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:50.541 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:50.535+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:50.541 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:50.536+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:50.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:50.619+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:50.757 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:50.753+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:51.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:51.088+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:51.387 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:51.382+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:51.476 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:51.471+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:51.581 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:51.575+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:51.582 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:51.577+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:51.607 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:51.602+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:51.749 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:51.744+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:52.108 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:52.101+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:52.404 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:52.399+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:52.458 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:52.452+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:52.580 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:52.574+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:52.585 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:52.580+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:52.586 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:52.581+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:52.732 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:52.727+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:53.106 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:53.102+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:53.378 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:53.373+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:53.485 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:53.480+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:53.546 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:53.542+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:53.580 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:53.575+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:53.581 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:53.577+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:53.737 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:53.732+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:54.158 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:54.153+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:54.425 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:54.421+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:54.516 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:54.512+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:54.522 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:54.518+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:54.558 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:54.553+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:54.558 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:54.555+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:54.787 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:54.782+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:55.143 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:55.139+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:55.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:55.409+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:55.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:55.470+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:55.498 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:55.494+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:55.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:55.547+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:55.554 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:55.551+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:55.794 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:55.789+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:56.107 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:56.104+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:56.442 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:39:56.435+0000 1694f700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.236545+0000 front 2019-12-09T18:38:41.971020+0000 (oldest deadline 2019-12-09T18:39:27.130954+0000) 2019-12-09T18:39:56.468 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:39:56.464+0000 1694f700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:41.678876+0000 front 2019-12-09T18:38:41.496533+0000 (oldest deadline 2019-12-09T18:39:25.492589+0000) 2019-12-09T18:39:56.534 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:56.528+0000 1694f700 -1 osd.3 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:42.734684+0000 front 2019-12-09T18:38:42.571506+0000 (oldest deadline 2019-12-09T18:39:25.404121+0000) 2019-12-09T18:39:56.534 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:56.530+0000 1694f700 -1 osd.3 183 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=3 ondisk+retry+write+known_if_redirected e181) 2019-12-09T18:39:56.540 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:56.535+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:56.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:39:56.804+0000 1694f700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:44.082016+0000 front 2019-12-09T18:38:44.186050+0000 (oldest deadline 2019-12-09T18:39:28.613147+0000) 2019-12-09T18:39:57.121 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:39:57.116+0000 1694f700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:38.662049+0000 front 2019-12-09T18:38:44.181179+0000 (oldest deadline 2019-12-09T18:39:23.290917+0000) 2019-12-09T18:39:57.346 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:39:57.341+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 46sec e183 v183) 2019-12-09T18:39:57.542 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:39:57.538+0000 1694f700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:38:43.449187+0000 front 2019-12-09T18:38:43.134243+0000 (oldest deadline 2019-12-09T18:39:28.425860+0000) 2019-12-09T18:39:58.263 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:58.259+0000 1694f700 -1 osd.3 184 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=4 ondisk+retry+write+known_if_redirected e183) 2019-12-09T18:39:59.274 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:39:59.270+0000 1694f700 -1 osd.3 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=4 ondisk+retry+write+known_if_redirected e183) 2019-12-09T18:40:00.398 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:40:00.395+0000 1694f700 -1 osd.3 185 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=4 ondisk+retry+write+known_if_redirected e183) 2019-12-09T18:40:34.207 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:34.194+0000 21452700 -1 reset not still connected to 0xed6b350 2019-12-09T18:40:34.207 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:34.195+0000 21452700 -1 reset not still connected to 0x1c914690 2019-12-09T18:40:37.592 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:40:37.550+0000 31f87700 -1 osd.3 187 build_incremental_map_msg missing incremental map 187 2019-12-09T18:40:37.936 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:40:37.861+0000 36790700 -1 osd.3 187 build_incremental_map_msg missing incremental map 187 2019-12-09T18:40:38.074 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:40:37.946+0000 37792700 -1 osd.3 187 build_incremental_map_msg missing incremental map 187 2019-12-09T18:40:38.685 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:38.660+0000 31f87700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:38.788 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:38.668+0000 30f85700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:38.819 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:38.690+0000 31f87700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:39.081 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:38.852+0000 34f8d700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:40.063 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:40.053+0000 30f85700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:41.053 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:40:41.013+0000 32f89700 -1 osd.4 188 build_incremental_map_msg missing incremental map 188 2019-12-09T18:40:56.027 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:40:56.022+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:40:56.604 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:40:56.598+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:40:56.868 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:40:56.865+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:40:57.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:40:57.053+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:40:57.653 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:40:57.647+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:40:57.923 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:40:57.918+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:40:58.099 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:40:58.093+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:40:58.105 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:40:58.100+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:40:58.489 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:40:58.484+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:40:58.633 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:40:58.627+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:40:58.865 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:40:58.860+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:40:58.880 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:40:58.875+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:40:59.066 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:40:59.061+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:40:59.124 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:40:59.119+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:40:59.509 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:40:59.504+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:40:59.621 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:40:59.616+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:40:59.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:40:59.824+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:40:59.882 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:40:59.877+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:00.081 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:00.077+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:00.178 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:00.173+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:00.482 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:00.479+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:00.632 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:00.627+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:00.869 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:00.864+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:00.908 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:00.901+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:01.131 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:01.125+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:01.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:01.154+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:01.516 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:01.511+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:01.632 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:01.627+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:01.889 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:01.884+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:01.958 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:01.953+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:02.102 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:02.092+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:02.197 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:02.192+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:02.544 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:02.538+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:02.650 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:02.645+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:02.866 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:19:54.715 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:02.903 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:19:54.753 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:02.919 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:02.914+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:02.920 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:19:54.745 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:02.926 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:19:54.750 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:02.926 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:19:54.751 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:02.926 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:02.922+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:02.928 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:19:54.753 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:03.035 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:19:54.860 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:03.088 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:19:54.913 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:03.124 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:19:54.974 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:41:03.125 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:03.121+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:03.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:03.160+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:03.570 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:03.565+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:03.619 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:03.614+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:03.896 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:03.891+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:03.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:03.923+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:04.147 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:04.142+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:04.181 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:04.174+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:04.572 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:04.567+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:04.623 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:04.618+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:04.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:04.872+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:04.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:04.911+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:05.195 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:05.188+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:05.200 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:05.194+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:05.540 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:05.535+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:05.640 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:05.635+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:05.889 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:05.885+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:05.940 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:05.935+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:06.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:06.200+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:06.253 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:06.248+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:06.510 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:06.505+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:06.685 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:06.680+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:06.868 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:06.864+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:06.904 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:06.898+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:07.233 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:07.230+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:07.258 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:07.253+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:07.528 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:07.522+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:07.692 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:07.688+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:07.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:07.898+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:07.936 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:07.930+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:08.191 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:08.185+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:08.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:08.298+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:08.560 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:08.554+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:08.711 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:08.707+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:08.882 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:08.876+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:08.961 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:08.956+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:09.238 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:09.233+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:09.316 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:09.310+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:09.544 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:09.539+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:09.664 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:09.660+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:09.927 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:09.923+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:10.005 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:10.000+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:10.245 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:10.241+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:10.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:10.332+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:10.538 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:10.531+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:10.676 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:10.672+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:10.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:10.921+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:11.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:11.053+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:11.212 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:11.208+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:11.299 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:11.294+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:11.544 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:11.538+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:11.724 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:11.719+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:11.846 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:11.842+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:11.882 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:11.877+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:12.030 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:12.023+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:12.202 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:12.197+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:12.319 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:12.315+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:12.543 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:12.538+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:12.679 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:12.674+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:12.805 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:12.801+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:12.850 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:12.845+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:13.018 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:13.014+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:13.190 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:13.187+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:13.372 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:13.365+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:13.573 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:13.568+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:13.655 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:13.651+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:13.825 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:13.820+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:13.826 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:13.822+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:14.064 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:14.056+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:14.163 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:14.159+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:14.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:14.407+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:14.587 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:14.581+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:14.659 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:14.654+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:14.820 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:14.816+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:14.856 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:14.844+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:15.048 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:15.042+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:15.134 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:15.129+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:15.463 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:15.457+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:15.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:15.601+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:15.689 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:15.683+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:15.789 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:15.783+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:15.790 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:15.785+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:15.845 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:15.841+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:16.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:16.094+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:16.102 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:16.093+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:16.417 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:16.412+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:16.636 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:16.631+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:16.690 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:16.686+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:16.823 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:16.819+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:16.831 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:16.825+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:16.832 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:16.827+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:17.128 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:17.122+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:17.144 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:17.141+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:17.432 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:17.427+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:17.660 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:17.656+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:17.669 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:17.663+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:17.835 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:17.826+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:17.836 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:17.831+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:17.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:17.866+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:18.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:18.109+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:18.166 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:18.160+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:18.440 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:18.434+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:18.652 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:18.647+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:18.701 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:18.684+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:18.808 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:18.803+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:18.812 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:18.808+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:18.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:18.829+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:19.124 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:19.120+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:19.204 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:19.200+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:19.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:19.454+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:19.667 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:19.663+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:19.685 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:19.679+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:19.769 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:19.763+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:19.770 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:19.765+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:19.878 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:19.873+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:20.175 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:20.170+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:20.176 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:20.170+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:20.451 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:20.445+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:20.680 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:20.676+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:20.724 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:20.718+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:20.762 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:20.756+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:20.763 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:20.758+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:20.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:20.884+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:21.157 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:21.154+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:21.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:21.205+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:21.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:21.423+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:21.671 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:21.665+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:21.691 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:21.686+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:21.810 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:21.804+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:21.813 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:21.809+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:21.869 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:21.864+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:22.144 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:22.140+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:22.217 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:22.211+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:22.437 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:22.432+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:22.705 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:22.700+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:22.719 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:22.715+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:22.789 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:22.784+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:22.790 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:22.786+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:22.918 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:22.914+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:23.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:23.134+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:23.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:23.234+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:23.439 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:23.434+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:23.680 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:23.676+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:23.729 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:23.724+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:23.759 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:23.755+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:23.762 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:23.758+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:23.899 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:23.894+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:24.120 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:24.114+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:24.275 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:24.269+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:24.469 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:24.463+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:24.656 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:24.651+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:24.694 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:24.688+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:24.747 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:24.740+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:24.748 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:24.743+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:24.938 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:24.935+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:25.077 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:25.073+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:25.264 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:25.258+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:25.523 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:25.517+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:25.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:25.613+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:25.693 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:25.688+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:25.769 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:25.763+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:25.770 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:25.765+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:25.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:25.970+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:26.109 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:26.104+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:26.287 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:26.281+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:26.523 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:26.517+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:26.623 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:26.619+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:26.721 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:26.716+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:26.749 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:26.745+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:26.750 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:26.746+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:26.999 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:26.993+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:27.118 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:27.113+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:27.270 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:27.265+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:27.533 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:27.527+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:27.590 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:27.583+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:27.736 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:27.730+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:27.742 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:27.736+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:27.743 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:27.738+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:27.907 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:41:27.902+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e188 v188) 2019-12-09T18:41:27.979 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:27.975+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:28.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:28.110+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:28.265 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:28.260+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:28.534 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:28.529+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:28.613 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:28.608+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:28.694 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:28.689+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:28.763 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:28.758+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:28.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:28.759+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:29.005 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:29.001+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:29.072 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:29.069+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:29.274 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:29.268+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:29.578 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:29.572+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:29.611 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:29.606+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:29.664 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:29.659+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:29.801 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:29.795+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:29.801 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:29.797+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:30.054 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:30.050+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:30.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:30.111+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:30.321 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:30.314+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:30.591 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:30.586+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:30.610 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:30.605+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:30.627 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:30.622+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:30.854 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:30.848+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:30.856 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:30.850+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:31.034 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:31.029+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:31.140 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:31.133+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:31.301 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:31.295+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:31.584 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:31.579+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:31.601 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:31.594+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:31.638 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:31.633+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:31.873 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:31.867+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:31.875 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:31.869+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:31.998 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:31.994+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:32.097 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:32.092+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:32.315 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:32.309+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:32.562 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:32.557+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:32.619 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:32.614+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:32.693 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:32.687+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:32.836 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:32.830+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:32.837 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:32.832+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:32.925 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:41:32.919+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e188 v188) 2019-12-09T18:41:33.011 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:33.007+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:33.078 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:33.074+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:33.300 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:33.295+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:33.531 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:33.527+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:33.659 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:33.653+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:33.668 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:33.663+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:33.824 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:33.818+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:33.825 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:33.820+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:33.983 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:33.979+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:34.062 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:34.057+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:34.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:34.299+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:34.581 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:34.577+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:34.629 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:34.624+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:34.640 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:34.634+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:34.802 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:34.794+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:34.802 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:34.797+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:34.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:34.969+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:35.045 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:35.040+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:35.339 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:35.333+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:35.616 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:35.612+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:35.642 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:35.637+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:35.654 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:35.649+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:35.789 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:35.783+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:35.790 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:35.786+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:35.932 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:35.926+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:36.029 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:36.024+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:36.330 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:36.324+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:36.589 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:36.584+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:36.641 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:36.635+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:36.642 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:36.638+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:36.792 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:36.787+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:36.795 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:36.791+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:36.913 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:36.909+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:36.993 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:36.988+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:37.287 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:37.282+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:37.583 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:37.579+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:37.659 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:37.653+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:37.670 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:37.665+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:37.826 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:37.820+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:37.827 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:37.823+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:37.933 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:37.924+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:37.947 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:41:37.940+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e188 v188) 2019-12-09T18:41:38.001 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:37.997+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:38.314 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:38.308+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:38.629 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:38.624+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:38.656 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:38.649+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:38.670 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:38.665+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:38.863 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:38.857+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:38.864 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:38.859+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:38.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:38.925+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:39.035 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:39.031+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:39.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:39.288+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:39.635 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:39.630+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:39.667 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:39.642+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:39.713 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:39.708+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:39.831 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:39.825+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:39.832 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:39.827+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:39.938 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:39.934+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:40.027 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:40.022+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:40.340 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:40.335+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:40.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:40.590+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:40.676 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:40.672+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:40.714 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:40.707+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:40.803 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:40.798+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:40.806 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:40.802+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:40.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:40.916+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:41.025 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:41.019+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:41.317 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:41.312+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:41.584 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:41.578+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:41.706 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:41.701+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:41.759 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:41.755+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:41.855 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:41.850+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:41.856 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:41.852+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:41.936 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:41.932+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:42.038 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:42.033+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:42.276 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:42.272+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:42.637 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:42.629+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:42.718 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:42.714+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:42.755 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:42.752+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:42.832 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:42.828+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:42.833 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:42.830+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:42.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:42.908+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:42.969 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:41:42.961+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e188 v188) 2019-12-09T18:41:43.030 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:43.026+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:43.243 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:43.239+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:43.654 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:43.650+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:43.680 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:43.676+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:43.731 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:43.726+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:43.815 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:43.810+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:43.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:43.812+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:43.955 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:43.951+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:44.068 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:44.063+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:44.271 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:44.267+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:44.646 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:44.641+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:44.704 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:44.699+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:44.708 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:44.703+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:44.823 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:44.816+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:44.824 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:44.820+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:44.909 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:44.904+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:45.097 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:45.093+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:45.272 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:45.266+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:45.631 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:45.625+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:45.704 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:45.700+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:45.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:45.721+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:45.811 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:45.805+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:45.814 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:45.807+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:45.956 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:45.952+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:46.105 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:46.100+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:46.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:46.299+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:46.653 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:46.647+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:46.684 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:46.680+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:46.687 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:46.682+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:46.860 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:46.854+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:46.861 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:46.856+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:46.941 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:46.936+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:47.117 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:47.112+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:41:47.267 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:41:47.261+0000 1694f700 -1 osd.7 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:10.219521+0000 front 2019-12-09T18:40:10.378202+0000 (oldest deadline 2019-12-09T18:40:56.081971+0000) 2019-12-09T18:41:47.616 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:41:47.611+0000 1694f700 -1 osd.5 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.453954+0000 front 2019-12-09T18:40:12.479605+0000 (oldest deadline 2019-12-09T18:40:57.566577+0000) 2019-12-09T18:41:47.657 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:41:47.651+0000 1694f700 -1 osd.6 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:12.676566+0000 front 2019-12-09T18:40:13.221120+0000 (oldest deadline 2019-12-09T18:40:57.320041+0000) 2019-12-09T18:41:47.727 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:41:47.722+0000 1694f700 -1 osd.1 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.371829+0000 front 2019-12-09T18:40:14.479855+0000 (oldest deadline 2019-12-09T18:40:56.045295+0000) 2019-12-09T18:41:47.897 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:47.891+0000 1694f700 -1 osd.4 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:40:35.781570+0000 (oldest deadline 2019-12-09T18:41:15.781570+0000) 2019-12-09T18:41:47.898 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:41:47.893+0000 1694f700 -1 osd.4 188 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e187) 2019-12-09T18:41:47.904 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:41:47.900+0000 1694f700 -1 osd.2 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:14.339053+0000 front 2019-12-09T18:40:14.313402+0000 (oldest deadline 2019-12-09T18:40:58.727531+0000) 2019-12-09T18:41:48.065 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:41:48.056+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e188 v188) 2019-12-09T18:41:48.130 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:41:48.124+0000 1694f700 -1 osd.0 188 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:40:11.226858+0000 front 2019-12-09T18:40:10.997263+0000 (oldest deadline 2019-12-09T18:40:55.574233+0000) 2019-12-09T18:42:16.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:16.439+0000 3578e700 -1 osd.3 194 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:16.485 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:16.461+0000 30f85700 -1 osd.3 194 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:16.601 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:16.571+0000 36790700 -1 osd.3 194 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:16.692 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:16.621+0000 3478c700 -1 osd.3 194 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:17.428 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:17.315+0000 124b0700 -1 osd.3 195 build_incremental_map_msg missing incremental map 195 2019-12-09T18:42:17.630 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:17.445+0000 37792700 -1 osd.3 195 build_incremental_map_msg missing incremental map 195 2019-12-09T18:42:18.029 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:18.000+0000 36f91700 -1 osd.3 195 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:18.046 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:18.000+0000 36f91700 -1 osd.3 195 build_incremental_map_msg missing incremental map 195 2019-12-09T18:42:21.540 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:21.528+0000 32f89700 -1 osd.3 195 build_incremental_map_msg missing incremental map 164 2019-12-09T18:42:22.120 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:22.112+0000 32f89700 -1 osd.3 195 build_incremental_map_msg missing incremental map 182 2019-12-09T18:42:22.120 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:22.113+0000 32f89700 -1 osd.3 195 build_incremental_map_msg missing incremental map 187 2019-12-09T18:42:22.132 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:22.128+0000 32f89700 -1 osd.3 195 build_incremental_map_msg missing incremental map 194 2019-12-09T18:42:22.137 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:22.129+0000 32f89700 -1 osd.3 195 build_incremental_map_msg missing incremental map 195 2019-12-09T18:42:48.032 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:48.028+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:49.034 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:49.031+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:50.023 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:50.020+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:51.009 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:51.004+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:51.968 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:51.964+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:52.950 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:52.946+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:53.911 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:53.908+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:54.903 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:54.900+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:55.890 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:55.886+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:56.906 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:56.903+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:57.870 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:57.866+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:58.902 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:58.898+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:42:59.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:42:59.865+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:00.835 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:00.831+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:01.875 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:01.871+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:02.906 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:02.902+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:03.958 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:03.954+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:05.013 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:05.009+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:06.062 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:06.059+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:07.101 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:07.098+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:08.151 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:08.147+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:09.163 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:09.159+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:10.151 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:10.146+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:11.084 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:22:02.933 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.089 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:22:02.939 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.105 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:22:02.930 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.110 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:22:02.935 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.110 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:22:02.935 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.116 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:22:02.941 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.179 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:11.175+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:11.273 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:22:03.123 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.281 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:22:03.106 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:11.288 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:22:03.113 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:43:12.150 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:12.146+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:13.118 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:13.113+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:13.807 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:13.802+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:14.153 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:14.148+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:14.281 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:14.276+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:14.803 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:14.797+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:15.113 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:15.107+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:15.114 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:15.109+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:15.297 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:15.292+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:15.360 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:15.354+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:15.785 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:15.778+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:16.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:16.058+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:16.089 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:16.083+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:16.090 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:16.085+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:16.310 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:16.306+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:16.413 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:16.407+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:16.571 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:16.566+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:16.757 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:16.752+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:17.104 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:17.097+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:17.123 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:17.117+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:17.124 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:17.119+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:17.273 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:17.268+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:17.415 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:17.409+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:17.552 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:17.547+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:17.577 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:17.570+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:17.785 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:17.780+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:18.109 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:18.103+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:18.150 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:18.143+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:18.150 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:18.145+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:18.296 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:18.291+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:18.418 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:18.411+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:18.541 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:18.537+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:18.628 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:18.623+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:18.778 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:18.773+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:19.147 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:19.142+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:19.148 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:19.142+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:19.148 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:19.144+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:19.260 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:19.256+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:19.377 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:19.370+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:19.585 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:19.579+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:19.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:19.590+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:19.741 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:19.735+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:20.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:20.179+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:20.193 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:20.188+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:20.193 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:20.189+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:20.215 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:20.211+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:20.411 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:20.405+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:20.593 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:20.587+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:20.631 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:20.626+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:20.741 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:20.735+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:21.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:21.186+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:21.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:21.187+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:21.221 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:21.217+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:21.257 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:21.254+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:21.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:21.409+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:21.558 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:21.553+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:21.604 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:21.599+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:21.793 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:21.787+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:22.155 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:22.149+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:22.155 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:22.150+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:22.192 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:22.187+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:22.273 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:22.266+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:22.420 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:22.415+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:22.521 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:22.516+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:22.594 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:22.589+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:22.802 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:22.797+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:23.196 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:23.191+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:23.209 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:23.201+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:23.211 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:23.202+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:23.249 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:23.244+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:23.441 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:23.435+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:23.492 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:23.483+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:23.624 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:23.619+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:23.809 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:23.803+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:24.218 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:24.213+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:24.243 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:24.238+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:24.257 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:24.251+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:24.257 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:24.252+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:24.463 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:24.459+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:24.478 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:24.474+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:24.587 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:24.582+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:24.791 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:24.786+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:25.224 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:25.218+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:25.240 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:25.235+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:25.281 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:25.276+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:25.282 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:25.277+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:25.488 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:25.484+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:25.510 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:25.505+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:25.562 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:25.556+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:25.815 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:25.810+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:26.198 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:26.192+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:26.222 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:26.217+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:26.260 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:26.254+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:26.261 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:26.256+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:26.506 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:26.499+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:26.529 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:26.508+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:26.568 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:26.563+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:26.844 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:26.839+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:27.163 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:27.158+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:27.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:27.261+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:27.273 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:27.268+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:27.274 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:27.270+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:27.466 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:27.460+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:27.508 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:27.503+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:27.584 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:27.580+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:27.797 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:27.792+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:28.187 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:28.181+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:28.276 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:28.272+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:28.328 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:28.322+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:28.329 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:28.324+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:28.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:28.444+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:28.534 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:28.528+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:28.628 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:28.624+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:28.833 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:28.819+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:29.146 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:29.140+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:29.258 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:29.254+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:29.372 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:29.367+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:29.372 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:29.368+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:29.463 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:29.458+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:29.562 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:29.556+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:29.663 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:29.658+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:29.810 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:29.806+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:30.134 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:30.128+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:30.255 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:30.249+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:30.327 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:30.321+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:30.327 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:30.323+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:30.437 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:30.431+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:30.533 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:30.525+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:30.672 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:30.666+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:30.780 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:30.772+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:31.117 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:31.111+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:31.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:31.221+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:31.327 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:31.321+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:31.328 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:31.324+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:31.459 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:31.454+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:31.494 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:31.489+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:31.657 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:31.653+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:31.774 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:31.770+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:32.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:32.099+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:32.200 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:32.195+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:32.306 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:32.301+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:32.307 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:32.303+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:32.485 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:32.480+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:32.496 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:32.482+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:32.634 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:32.628+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:32.781 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:32.777+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:33.061 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:33.057+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:33.178 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:33.173+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:33.279 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:33.275+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:33.280 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:33.276+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:33.477 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:33.472+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:33.526 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:33.522+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:33.638 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:33.633+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:33.791 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:33.786+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:34.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:34.056+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:34.183 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:34.178+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:34.236 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:34.231+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:34.237 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:34.233+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:34.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:34.442+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:34.490 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:34.485+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:34.649 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:34.644+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:34.836 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:34.831+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:35.045 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:35.041+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:35.205 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:35.200+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:35.206 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:35.202+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:35.220 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:35.215+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:35.465 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:35.459+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:35.507 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:35.500+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:35.689 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:35.684+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:35.843 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:35.838+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:36.024 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:36.019+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:36.242 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:36.238+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:36.242 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:36.239+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:36.263 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:36.259+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:36.454 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:36.451+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:36.521 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:36.515+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:36.736 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:36.732+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:36.823 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:36.819+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:37.020 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:37.016+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:37.232 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:37.227+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:37.274 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:37.270+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:37.275 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:37.271+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:37.423 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:37.419+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:37.557 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:37.552+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:37.757 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:37.752+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:37.856 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:37.852+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:38.006 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:38.003+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:38.204 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:38.199+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:38.257 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:38.252+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:38.258 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:38.254+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:38.440 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:38.435+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:38.588 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:38.583+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:38.715 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:38.710+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:38.909 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:38.905+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:39.053 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:39.048+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:39.170 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:39.165+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:39.303 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:39.299+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:39.304 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:39.300+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:39.483 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:39.478+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:39.568 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:39.563+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:39.676 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:39.672+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:39.870 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:39.866+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:40.049 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:40.043+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:40.210 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:40.204+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:40.322 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:40.317+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:40.322 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:40.319+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:40.521 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:40.517+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:40.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:40.604+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:40.700 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:40.696+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:40.846 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:40.842+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:41.064 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:41.059+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:41.229 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:41.224+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:41.328 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:41.322+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:41.328 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:41.323+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:41.486 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:41.481+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:41.573 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:41.568+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:41.670 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:41.665+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:41.856 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:41.851+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:42.087 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:42.080+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:42.207 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:42.202+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:42.360 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:42.354+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:42.360 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:42.356+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:42.471 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:42.467+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:42.551 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:42.546+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:42.649 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:42.644+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:42.816 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:42.812+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:43.081 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:43.076+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:43.248 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:43.241+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:43.408 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:43.403+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:43.409 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:43.404+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:43.507 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:43.503+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:43.508 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:43.504+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:43.643 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:43.639+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:43.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:43.791+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:44.074 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:44.070+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:44.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:44.260+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:44.404 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:44.399+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:44.405 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:44.401+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:44.513 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:44.509+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:44.528 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:44.522+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:44.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:44.599+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:44.828 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:44.823+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:45.110 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:45.106+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:45.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:45.267+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:45.421 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:45.416+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:45.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:45.418+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:45.472 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:45.467+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:45.524 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:45.518+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:45.572 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:45.567+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:45.785 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:45.781+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:46.156 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:46.151+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:46.299 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:46.294+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:46.428 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:46.422+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:46.428 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:46.424+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:46.500 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:46.495+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:46.529 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:46.524+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:46.598 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:46.593+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:46.827 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:46.823+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:47.161 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:47.156+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:47.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:47.261+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:47.451 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:47.446+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:47.452 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:47.448+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:47.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:47.545+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:47.566 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:47.563+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:47.646 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:47.641+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:47.800 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:47.795+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:48.140 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:48.134+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:48.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:48.273+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:48.414 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:48.408+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:48.414 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:48.410+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:48.529 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:43:48.523+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 45sec e195 v195) 2019-12-09T18:43:48.545 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:48.542+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:48.595 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:48.591+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:48.664 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:48.661+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:48.805 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:48.800+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:49.173 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:49.159+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:49.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:49.260+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:49.443 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:49.437+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:49.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:49.439+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:49.558 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:49.550+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:49.602 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:49.598+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:49.708 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:49.703+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:49.801 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:49.797+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:50.161 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:50.156+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:50.286 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:50.281+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:50.477 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:50.472+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:50.478 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:50.474+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:50.539 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:50.532+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:50.622 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:50.617+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:50.729 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:50.724+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:50.799 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:50.794+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:51.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:51.161+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:51.327 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:51.322+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:51.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:51.493+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:51.499 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:51.495+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:51.569 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:51.563+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:51.596 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:51.592+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:51.738 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:51.734+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:51.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:51.807+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:52.126 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:52.122+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:52.304 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:52.299+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:52.518 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:52.513+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:52.548 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:52.514+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:52.554 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:52.550+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:52.611 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:52.607+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:52.760 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:52.755+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:52.787 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:43:52.783+0000 1694f700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:33.902663+0000 front 2019-12-09T18:42:29.648220+0000 (oldest deadline 2019-12-09T18:43:13.521054+0000) 2019-12-09T18:43:53.089 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:43:53.084+0000 1694f700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.811531+0000 front 2019-12-09T18:42:35.548867+0000 (oldest deadline 2019-12-09T18:43:15.483774+0000) 2019-12-09T18:43:53.292 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:43:53.287+0000 1694f700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.550260+0000 front 2019-12-09T18:42:34.233252+0000 (oldest deadline 2019-12-09T18:43:14.187617+0000) 2019-12-09T18:43:53.554 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:43:53.550+0000 1694f700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:34.928825+0000 front 2019-12-09T18:42:30.672375+0000 (oldest deadline 2019-12-09T18:43:14.891451+0000) 2019-12-09T18:43:53.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:53.551+0000 1694f700 -1 osd.3 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:30.672238+0000 front 2019-12-09T18:42:34.520776+0000 (oldest deadline 2019-12-09T18:43:14.320914+0000) 2019-12-09T18:43:53.556 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:53.553+0000 1694f700 -1 osd.3 195 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:53.584 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:43:53.579+0000 1694f700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:32.787626+0000 front 2019-12-09T18:42:32.952404+0000 (oldest deadline 2019-12-09T18:43:17.350095+0000) 2019-12-09T18:43:53.616 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:43:53.600+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 45sec e195 v195) 2019-12-09T18:43:53.777 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:43:53.764+0000 1694f700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:42:35.818787+0000 front 2019-12-09T18:42:35.839214+0000 (oldest deadline 2019-12-09T18:43:16.302506+0000) 2019-12-09T18:43:54.617 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:54.613+0000 1694f700 -1 osd.3 196 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:55.673 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:55.669+0000 1694f700 -1 osd.3 197 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:43:57.023 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:43:56.966+0000 1694f700 -1 osd.3 197 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2:2c9d2974:::7b90ce1f-2e6d-4ddb-b794-a516be24e604.4458.631_kappa%2f40e0614c65d44030a950687ac17f6066:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~0,setxattr user.rgw.manifest (367) in=384b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] RETRY=6 ondisk+retry+write+known_if_redirected e193) 2019-12-09T18:44:28.130 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:28.111+0000 35f8f700 -1 osd.4 200 build_incremental_map_msg missing incremental map 200 2019-12-09T18:44:28.162 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:28.119+0000 30f85700 -1 osd.4 200 build_incremental_map_msg missing incremental map 200 2019-12-09T18:44:28.202 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:28.187+0000 30f85700 -1 osd.4 200 build_incremental_map_msg missing incremental map 200 2019-12-09T18:44:28.649 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:28.574+0000 32f89700 -1 osd.4 200 build_incremental_map_msg missing incremental map 200 2019-12-09T18:44:28.886 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:28.867+0000 34f8d700 -1 osd.4 200 build_incremental_map_msg missing incremental map 200 2019-12-09T18:44:52.406 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:52.400+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:52.770 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:52.764+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:53.420 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:53.414+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:53.822 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:53.816+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:54.069 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:54.064+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:54.429 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:54.425+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:54.430 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:54.425+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:54.813 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:54.808+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:55.075 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:44:55.071+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:44:55.103 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:55.098+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:55.400 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:55.395+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:55.450 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:55.445+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:55.795 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:55.789+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:56.058 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:44:56.053+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:44:56.128 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:56.123+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:56.311 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:44:56.308+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:44:56.406 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:56.401+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:56.449 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:56.442+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:56.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:56.824+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:57.014 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:44:57.010+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:44:57.126 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:57.120+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:57.332 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:44:57.326+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:44:57.364 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:57.359+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:57.480 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:57.475+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:57.817 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:57.812+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:57.983 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:44:57.979+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:44:57.997 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:57.993+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:44:58.141 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:58.135+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:58.340 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:44:58.336+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:44:58.349 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:58.340+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:58.455 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:58.450+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:58.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:58.825+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:58.991 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:58.986+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:44:59.019 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:44:59.014+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:44:59.094 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:44:59.089+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:44:59.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:44:59.309+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:44:59.326 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:44:59.321+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:44:59.422 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:44:59.417+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:44:59.858 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:44:59.853+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:44:59.977 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:44:59.973+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:00.045 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:00.039+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:00.104 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:00.100+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:00.329 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:00.326+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:00.354 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:00.349+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:00.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:00.372+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:00.886 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:00.880+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:00.947 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:00.944+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:01.039 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:01.035+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:01.124 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:01.119+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:01.335 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:01.332+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:01.336 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:01.332+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:01.414 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:01.410+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:01.895 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:01.890+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:01.998 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:01.994+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:02.083 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:02.079+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:02.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:02.128+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:02.316 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:02.312+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:02.365 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:02.361+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:02.443 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:02.438+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:02.852 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:02.847+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:03.042 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:03.038+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:03.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:03.070+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:03.182 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:03.176+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:03.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:03.288+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:03.320 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:03.316+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:03.420 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:03.414+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:03.836 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:03.830+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:04.051 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:04.047+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:04.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:04.092+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:04.196 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:04.190+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:04.273 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:04.268+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:04.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:04.282+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:04.435 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:04.429+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:04.854 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:04.849+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:05.051 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:05.044+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:05.052 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:05.047+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:05.077 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:05.073+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:05.188 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:05.184+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:05.252 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:05.248+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:05.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:05.290+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:05.490 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:05.482+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:05.815 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:05.809+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:06.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:06.068+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:06.078 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:06.074+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:06.113 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:06.108+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:06.214 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:06.209+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:06.270 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:06.262+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:06.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:06.294+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:06.486 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:06.480+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:06.848 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:06.843+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:07.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:07.058+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:07.067 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:07.063+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:07.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:07.100+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:07.211 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:07.205+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:07.317 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:07.313+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:07.328 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:07.325+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:07.447 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:07.442+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:07.880 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:07.875+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:08.084 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:08.077+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:08.085 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:08.080+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:08.087 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:08.082+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:08.205 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:08.184+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:08.331 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:08.326+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:08.362 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:08.357+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:08.479 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:08.474+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:08.843 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:08.836+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:09.044 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:09.038+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:09.045 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:09.040+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:09.074 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:09.069+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:09.239 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:09.233+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:09.350 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:09.346+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:09.377 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:09.372+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:09.453 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:09.447+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:09.809 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:09.803+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:10.006 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:09.999+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:10.007 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:10.002+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:10.127 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:10.122+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:10.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:10.266+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:10.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:10.396+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:10.414 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:10.409+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:10.506 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:10.501+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:10.818 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:10.813+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:11.031 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:11.027+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:11.035 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:11.032+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:11.171 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:11.167+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:11.326 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:11.321+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:11.379 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:11.376+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:11.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:11.439+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:11.491 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:11.486+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:11.859 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:11.855+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:12.060 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:12.054+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:12.061 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:12.057+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:12.152 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:12.148+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:12.337 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:12.332+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:12.348 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:12.343+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:12.399 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:12.394+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:12.540 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:12.535+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:12.870 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:12.863+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:13.020 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:13.015+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:13.021 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:13.016+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:13.194 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:13.190+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:13.346 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:13.340+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:13.368 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:13.364+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:13.420 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:13.417+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:13.496 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:13.490+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:13.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:13.868+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:14.023 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:14.014+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:14.024 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:14.018+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:14.231 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:14.227+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:14.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:14.348+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:14.358 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:14.355+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:14.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:14.398+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:14.525 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:14.520+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:14.885 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:14.879+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:15.048 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:15.041+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:15.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:15.045+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:15.193 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:15.190+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:15.368 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:15.362+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:15.372 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:15.368+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:15.411 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:15.407+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:15.502 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:15.496+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:15.923 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:15.917+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:16.093 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:16.087+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:16.094 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:16.089+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:16.216 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:16.211+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:16.367 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:16.363+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:16.377 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:16.373+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:16.395 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:16.389+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:16.491 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:16.486+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:16.951 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:16.945+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:17.097 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:17.091+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:17.098 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:17.093+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:17.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:17.180+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:17.336 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:17.332+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:17.370 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:17.367+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:17.431 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:17.422+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:17.545 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:17.539+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:17.941 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:17.936+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:18.096 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:18.091+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:18.098 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:18.095+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:18.223 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:18.219+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:18.344 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:18.338+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:18.392 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:18.386+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:18.485 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:18.475+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:18.551 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:18.545+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:18.898 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:18.893+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:19.123 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:19.119+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:19.124 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:19.120+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:19.216 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:19.212+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:19.345 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:19.341+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:19.369 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:19.365+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:19.472 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:19.466+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:19.603 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:19.597+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:19.903 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:19.897+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:20.115 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:20.109+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:20.116 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:20.111+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:20.207 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:20.203+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:20.368 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:20.363+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:20.396 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:20.391+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:20.495 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:20.489+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:20.629 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:20.624+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:20.861 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:20.856+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:21.104 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:21.098+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:21.105 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:21.100+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:21.222 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:21.218+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:21.376 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:21.372+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:21.391 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:21.387+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:21.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:21.464+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:21.682 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:21.676+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:21.902 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:21.896+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:22.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:22.068+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:22.080 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:22.070+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:22.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:22.231+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:22.392 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:22.387+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:22.429 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:22.424+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:22.435 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:22.428+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:22.682 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:22.676+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:22.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:22.922+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:23.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:23.041+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:23.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:23.043+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:23.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:23.201+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:23.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:23.393+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:23.419 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:23.414+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:23.460 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:23.455+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:23.685 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:23.679+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:23.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:23.889+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:24.003 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:45:23.996+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e201 v201) 2019-12-09T18:45:24.101 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:24.095+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:24.101 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:24.097+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:24.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:24.201+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:24.405 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:24.399+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:24.424 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:24.418+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:24.428 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:24.423+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:24.681 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:24.674+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:24.931 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:24.926+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:25.055 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:25.051+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:25.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:25.054+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:25.203 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:25.200+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:25.368 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:25.365+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:25.384 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:25.380+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:25.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:25.400+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:25.642 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:25.636+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:25.967 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:25.961+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:26.040 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:26.033+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:26.041 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:26.036+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:26.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:26.162+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:26.395 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:26.391+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:26.405 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:26.399+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:26.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:26.441+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:26.662 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:26.657+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:27.012 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:27.003+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:27.038 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:27.033+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:27.041 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:27.037+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:27.190 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:27.185+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:27.406 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:27.401+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:27.427 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:27.422+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:27.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:27.422+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:27.625 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:27.619+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:28.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:28.052+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:28.088 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:28.083+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:28.091 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:28.087+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:28.206 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:28.202+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:28.402 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:28.398+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:28.407 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:28.403+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:28.456 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:28.451+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:28.656 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:28.651+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:29.012 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:29.007+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:29.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:29.058+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:29.067 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:29.063+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:29.113 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:45:29.100+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e201 v201) 2019-12-09T18:45:29.162 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:29.158+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:29.404 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:29.399+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:29.422 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:29.416+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:29.436 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:29.430+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:29.685 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:29.680+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:30.043 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:30.037+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:30.073 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:30.068+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:30.077 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:30.073+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:30.125 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:30.121+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:30.429 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:30.423+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:30.433 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:30.429+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:30.449 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:30.443+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:30.684 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:30.679+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:31.087 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:31.082+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:31.096 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:31.091+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:31.100 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:31.096+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:31.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:31.161+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:31.393 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:31.387+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:31.394 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:31.390+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:31.492 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:31.489+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:31.679 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:31.674+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:32.063 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:32.058+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:32.086 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:32.081+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:32.090 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:32.086+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:32.127 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:32.122+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:32.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:32.350+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:32.355 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:32.349+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:32.474 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:32.471+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:32.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:32.683+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:33.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:33.074+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:33.085 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:33.081+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:33.124 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:33.119+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:33.124 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:33.121+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:33.321 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:33.317+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:33.322 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:33.316+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:33.446 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:33.441+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:33.735 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:33.729+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:34.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:34.056+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:34.120 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:34.114+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:34.125 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:34.120+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:34.128 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:34.125+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:34.287 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:45:34.282+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e201 v201) 2019-12-09T18:45:34.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:34.327+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:34.340 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:34.335+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:34.366 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:24:26.216 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.388 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:24:26.213 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.390 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:24:26.215 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.400 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:24:26.250 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.421 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:24:26.240 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.458 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:24:26.282 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:34.468+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:34.548 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:24:26.373 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.595 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:24:26.419 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.676 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:24:26.526 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:45:34.769 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:34.764+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:35.066 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:35.061+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:35.121 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:35.116+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:35.149 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:35.143+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:35.150 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:35.145+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:35.305 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:35.299+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:35.332 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:35.328+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:35.472 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:35.466+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:35.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:35.806+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:36.043 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:36.038+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:36.131 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:36.125+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:36.132 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:36.127+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:36.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:36.160+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:36.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:36.299+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:36.342 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:36.336+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:36.445 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:36.440+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:36.864 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:36.858+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:37.023 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:37.017+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:37.153 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:37.147+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:37.168 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:37.162+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:37.212 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:37.207+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:37.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:37.286+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:37.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:37.384+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:37.411 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:37.406+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:37.864 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:37.858+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:38.002 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:37.996+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:38.191 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:38.184+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:38.191 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:38.187+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:38.230 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:38.224+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:38.344 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:38.339+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:38.381 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:38.376+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:38.420 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:38.415+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:38.866 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:45:38.860+0000 1694f700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:07.557745+0000 front 2019-12-09T18:44:08.038958+0000 (oldest deadline 2019-12-09T18:44:51.545832+0000) 2019-12-09T18:45:38.971 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:45:38.965+0000 1694f700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.786026+0000 front 2019-12-09T18:44:09.633266+0000 (oldest deadline 2019-12-09T18:44:52.468931+0000) 2019-12-09T18:45:39.207 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:45:39.202+0000 1694f700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.727820+0000 front 2019-12-09T18:44:11.024570+0000 (oldest deadline 2019-12-09T18:44:54.707712+0000) 2019-12-09T18:45:39.245 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:39.238+0000 1694f700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:44:24.232771+0000 (oldest deadline 2019-12-09T18:45:04.232771+0000) 2019-12-09T18:45:39.245 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:39.241+0000 1694f700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-12-09T18:45:39.317 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:45:39.307+0000 1694f700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:10.726306+0000 front 2019-12-09T18:44:11.031602+0000 (oldest deadline 2019-12-09T18:44:55.336717+0000) 2019-12-09T18:45:39.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:45:39.341+0000 1694f700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.121835+0000 front 2019-12-09T18:44:09.338167+0000 (oldest deadline 2019-12-09T18:44:54.399142+0000) 2019-12-09T18:45:39.373 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:45:39.363+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e201 v201) 2019-12-09T18:45:39.384 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:45:39.379+0000 1694f700 -1 osd.6 201 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:44:09.947425+0000 front 2019-12-09T18:44:09.975187+0000 (oldest deadline 2019-12-09T18:44:54.014999+0000) 2019-12-09T18:45:39.916 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:45:39.789+0000 36f91700 -1 osd.4 202 build_incremental_map_msg missing incremental map 200 2019-12-09T18:46:09.762 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:46:09.558+0000 3638f700 -1 osd.2 205 build_incremental_map_msg missing incremental map 205 2019-12-09T18:46:09.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:46:09.709+0000 37b92700 -1 osd.2 205 build_incremental_map_msg missing incremental map 205 2019-12-09T18:46:09.886 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:46:09.737+0000 38393700 -1 osd.2 205 build_incremental_map_msg missing incremental map 205 2019-12-09T18:46:11.451 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:11.417+0000 31786700 -1 osd.3 206 build_incremental_map_msg missing incremental map 206 2019-12-09T18:46:11.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:11.428+0000 31786700 -1 osd.3 206 build_incremental_map_msg missing incremental map 206 2019-12-09T18:46:12.175 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:12.152+0000 37792700 -1 osd.3 206 build_incremental_map_msg missing incremental map 206 2019-12-09T18:46:14.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:14.450+0000 32f89700 -1 osd.3 207 build_incremental_map_msg missing incremental map 206 2019-12-09T18:46:45.426 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:45.422+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:46.404 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:46.400+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:47.405 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:47.398+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:48.439 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:48.435+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:49.491 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:49.486+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:50.505 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:50.500+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:51.526 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:51.522+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:52.578 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:52.573+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:53.602 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:53.598+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:54.618 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:54.614+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:55.637 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:55.633+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:56.618 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:56.613+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:57.639 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:57.634+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:58.601 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:58.597+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:46:59.595 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:46:59.590+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:00.605 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:00.601+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:01.643 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:01.638+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:02.663 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:02.659+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:03.235 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:25:55.084 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.245 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:25:55.093 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.251 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:25:55.075 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.251 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:25:55.076 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.254 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:25:55.079 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.256 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:25:55.081 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.373 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:25:55.197 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.381 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:25:55.203 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.385 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:25:55.232 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:47:03.638 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:03.633+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:04.685 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:04.681+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:05.723 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:05.719+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:06.731 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:06.726+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:07.729 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:07.724+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:08.696 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:08.691+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:09.719 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:09.715+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:09.886 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:09.883+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:10.690 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:10.685+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:10.691 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:10.687+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:10.862 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:10.858+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:11.147 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:11.142+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:11.741 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:11.732+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:11.742 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:11.735+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:11.859 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:11.854+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:11.911 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:11.907+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:12.111 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:12.105+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:12.764 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:12.759+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:12.765 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:12.761+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:12.907 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:12.902+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:12.947 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:12.943+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:13.069 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:13.063+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:13.075 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:13.070+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:13.272 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:13.267+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:13.514 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:13.509+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:13.816 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:13.812+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:13.818 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:13.815+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:13.910 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:13.905+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:13.963 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:13.957+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:14.065 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:14.060+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:14.094 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:14.089+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:14.314 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:14.309+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:14.492 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:14.489+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:14.821 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:14.813+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:14.821 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:14.814+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:14.897 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:14.891+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:14.989 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:14.984+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:15.071 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:15.066+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:15.103 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:15.097+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:15.328 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:15.324+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:15.527 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:15.523+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:15.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:15.864+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:15.870 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:15.866+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:15.938 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:15.935+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:16.026 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:16.021+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:16.036 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:16.031+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:16.065 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:16.061+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:16.284 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:16.279+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:16.496 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:16.492+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:16.872 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:16.867+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:16.873 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:16.869+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:16.966 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:16.961+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:17.048 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:17.045+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:17.051 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:17.048+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:17.075 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:17.072+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:17.323 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:17.318+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:17.453 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:17.449+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:17.906 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:17.902+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:17.907 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:17.903+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:18.006 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:18.001+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:18.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:18.002+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:18.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:18.039+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:18.061 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:18.058+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:18.321 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:18.316+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:18.445 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:18.439+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:18.936 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:18.932+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:18.940 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:18.935+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:19.025 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:19.020+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:19.051 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:19.046+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:19.065 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:19.060+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:19.100 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:19.095+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:19.347 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:19.342+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:19.489 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:19.484+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:19.913 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:19.908+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:19.916 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:19.913+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:20.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:20.035+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:20.049 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:20.044+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:20.100 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:20.095+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:20.112 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:20.107+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:20.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:20.328+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:20.525 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:20.521+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:20.965 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:20.960+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:20.965 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:20.962+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:20.998 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:20.994+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:21.020 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:21.015+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:21.108 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:21.104+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:21.138 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:21.134+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:21.376 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:21.372+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:21.484 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:21.480+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:21.924 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:21.918+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:21.924 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:21.920+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:21.954 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:21.949+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:22.053 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:22.048+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:22.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:22.096+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:22.141 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:22.134+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:22.337 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:22.332+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:22.453 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:22.448+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:22.925 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:22.919+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:22.925 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:22.922+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:22.973 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:22.968+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:23.085 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:23.079+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:23.102 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:23.097+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:23.168 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:23.164+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:23.329 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:23.324+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:23.507 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:23.499+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:23.946 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:23.942+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:23.947 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:23.943+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:24.011 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:24.006+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:24.045 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:24.041+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:24.067 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:24.062+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:24.190 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:24.186+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:24.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:24.347+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:24.489 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:24.484+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:24.950 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:24.946+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:24.950 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:24.947+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:24.984 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:24.980+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:25.056 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:25.052+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:25.087 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:25.083+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:25.145 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:25.140+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:25.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:25.349+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:25.472 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:25.468+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:25.987 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:25.981+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:25.988 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:25.984+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:26.029 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:26.025+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:26.056 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:26.052+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:26.060 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:26.056+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:26.131 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:26.127+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:26.397 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:26.392+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:26.489 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:26.482+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:26.998 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:26.993+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:27.001 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:26.997+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:27.002 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:26.999+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:27.021 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:27.017+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:27.094 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:27.089+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:27.164 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:27.159+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:27.417 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:27.414+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:27.449 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:27.445+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:27.961 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:27.956+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:28.013 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:28.009+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:28.030 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:28.024+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:28.030 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:28.027+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:28.070 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:28.064+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:28.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:28.133+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:28.469 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:28.465+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:28.470 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:28.466+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:28.918 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:28.913+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:29.046 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:29.041+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:29.079 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:29.075+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:29.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:29.079+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:29.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:29.097+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:29.126 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:29.123+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:29.471 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:29.467+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:29.483 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:29.478+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:29.913 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:29.909+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:30.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:30.075+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:30.091 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:30.086+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:30.127 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:30.123+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:30.127 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:30.124+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:30.140 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:30.137+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:30.454 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:30.450+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:30.498 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:30.494+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:30.934 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:30.929+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:31.045 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:31.040+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:31.080 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:31.075+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:31.135 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:31.130+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:31.136 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:31.133+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:31.178 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:31.174+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:31.423 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:31.419+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:31.517 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:31.512+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:31.907 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:31.900+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:32.087 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:32.081+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:32.116 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:32.111+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:32.188 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:32.184+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:32.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:32.188+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:32.231 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:32.227+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:32.403 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:32.398+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:32.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:32.565+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:32.914 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:32.908+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:33.088 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:33.083+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:33.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:33.116+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:33.223 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:33.220+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:33.224 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:33.222+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:33.241 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:33.236+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:33.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:33.412+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:33.614 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:33.610+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:33.875 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:33.869+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:34.114 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:34.109+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:34.142 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:34.132+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:34.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:34.222+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:34.259 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:34.253+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:34.260 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:34.255+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:34.437 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:34.432+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:34.573 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:34.568+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:34.864 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:34.860+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:35.148 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:35.144+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:35.150 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:35.146+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:35.189 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:35.185+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:35.221 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:35.216+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:35.222 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:35.218+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:35.460 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:35.456+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:35.555 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:35.551+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:35.885 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:35.880+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:36.129 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:36.124+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:36.141 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:36.136+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:36.191 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:36.185+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:36.232 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:36.225+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:36.235 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:36.230+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:36.494 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:36.489+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:36.548 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:36.543+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:36.926 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:36.922+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:37.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:37.114+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:37.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:37.131+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:37.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:37.233+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:37.287 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:37.282+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:37.287 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:37.284+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:37.451 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:37.446+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:37.596 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:37.591+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:37.970 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:37.965+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:38.105 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:38.100+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:38.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:38.140+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:38.281 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:38.276+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:38.339 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:38.333+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:38.341 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:38.338+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:38.470 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:38.465+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:38.570 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:38.566+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:39.014 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:39.009+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:39.094 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:39.090+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:39.099 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:39.095+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:39.275 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:39.269+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:39.322 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:39.318+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:39.323 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:39.319+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:39.429 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:39.423+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:39.596 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:39.592+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:40.019 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:40.015+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:40.052 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:40.045+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:40.093 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:40.089+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:40.276 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:40.272+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:40.343 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:40.335+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:40.344 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:40.339+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:40.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:40.411+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:40.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:40.612+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:40.974 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:40.968+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:41.024 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:41.019+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:41.088 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:41.084+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:41.293 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:41.290+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:41.294 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:41.290+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:41.300 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:41.296+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:41.393 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:41.389+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:41.628 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:41.618+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:41.986 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:41.979+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:41.989 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:41.985+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:42.079 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:42.075+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:42.263 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:42.257+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:42.298 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:42.292+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:42.299 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:42.295+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:42.435 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:42.431+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:42.633 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:42.627+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:43.014 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:43.009+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:43.036 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:43.032+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:43.133 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:43.129+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:43.282 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:43.277+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:43.398 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:43.279+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:43.398 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:43.281+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:43.462 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:43.458+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:43.653 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:43.649+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:44.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:44.053+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:44.067 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:44.062+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:47:44.105 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:47:44.100+0000 1694f700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:25.299165+0000 front 2019-12-09T18:46:25.323566+0000 (oldest deadline 2019-12-09T18:47:11.152380+0000) 2019-12-09T18:47:44.267 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:44.262+0000 1694f700 -1 osd.3 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.790461+0000 front 2019-12-09T18:46:27.857777+0000 (oldest deadline 2019-12-09T18:47:10.534678+0000) 2019-12-09T18:47:44.268 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:47:44.264+0000 1694f700 -1 osd.3 207 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:113342 2.34s0 2.2e94b934 (undecoded) ondisk+retry+write+known_if_redirected e205) 2019-12-09T18:47:44.286 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:47:44.281+0000 1694f700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.872521+0000 front 2019-12-09T18:46:25.374194+0000 (oldest deadline 2019-12-09T18:47:09.420888+0000) 2019-12-09T18:47:44.507 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:47:44.503+0000 1694f700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:29.570392+0000 front 2019-12-09T18:46:29.572085+0000 (oldest deadline 2019-12-09T18:47:12.310555+0000) 2019-12-09T18:47:44.653 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:47:44.649+0000 1694f700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.716137+0000 front 2019-12-09T18:46:27.866171+0000 (oldest deadline 2019-12-09T18:47:12.878122+0000) 2019-12-09T18:47:44.920 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:47:44.913+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e207 v207) 2019-12-09T18:47:45.020 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:47:45.013+0000 1694f700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:27.447030+0000 front 2019-12-09T18:46:28.714954+0000 (oldest deadline 2019-12-09T18:47:10.415391+0000) 2019-12-09T18:47:45.079 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:47:45.074+0000 1694f700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:46:28.145587+0000 front 2019-12-09T18:46:29.131352+0000 (oldest deadline 2019-12-09T18:47:12.794809+0000) 2019-12-09T18:48:15.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:15.698+0000 1694f700 -1 osd.2 209 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:114323 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821402496 gen 84] snapc 0=[] ondisk+write+known_if_redirected e208) 2019-12-09T18:48:23.112 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:22.972+0000 35f8f700 -1 osd.6 211 build_incremental_map_msg missing incremental map 211 2019-12-09T18:48:23.156 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:23.085+0000 38f95700 -1 osd.6 211 build_incremental_map_msg missing incremental map 211 2019-12-09T18:48:23.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:23.107+0000 33f8b700 -1 osd.6 211 build_incremental_map_msg missing incremental map 211 2019-12-09T18:48:23.169 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:23.128+0000 33f8b700 -1 osd.6 211 build_incremental_map_msg missing incremental map 211 2019-12-09T18:48:43.546 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:43.541+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:44.592 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:44.586+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:44.674 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:44.667+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:44.701 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:44.695+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:44.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:44.848+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:45.183 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:45.180+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:45.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:45.542+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:45.709 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:45.703+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:45.725 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:45.719+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:45.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:45.827+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:46.225 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:46.219+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:46.598 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:46.591+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:46.687 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:46.681+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:46.693 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:46.688+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:46.812 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:46.807+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:47.207 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:47.202+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:47.633 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:47.628+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:47.659 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:47.654+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:47.673 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:47.668+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:47.780 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:47.776+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:48.192 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:48.188+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:48.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:48.221+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:48.640 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:48.637+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:48.641 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:48.636+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:48.666 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:48.661+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:48.754 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:48.749+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:49.217 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:49.209+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:49.243 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:49.237+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:49.634 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:49.629+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:49.675 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:49.664+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:49.699 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:49.692+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:49.718 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:49.715+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:50.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:50.166+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:50.203 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:50.198+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:50.593 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:50.586+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:50.635 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:50.630+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:50.685 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:50.678+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:50.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:50.740+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:51.169 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:51.166+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:51.258 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:51.253+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:51.573 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:51.569+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:51.625 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:51.620+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:51.695 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:51.690+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:51.724 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:51.720+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:52.168 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:52.164+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:52.221 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:52.215+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:52.604 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:52.598+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:52.625 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:52.619+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:52.707 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:52.701+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:52.716 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:52.712+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:53.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:53.165+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:53.232 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:53.227+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:53.613 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:53.607+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:53.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:53.667+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:53.673 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:53.669+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:53.721 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:53.716+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:54.145 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:54.138+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:54.225 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:54.220+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:54.575 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:54.569+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:54.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:54.669+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:54.722 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:54.717+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:54.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:54.751+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:55.115 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:55.109+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:55.220 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:55.214+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:55.624 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:55.617+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:55.698 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:55.694+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:55.713 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:55.709+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:55.751 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:55.745+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:56.135 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:56.130+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:56.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:56.230+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:56.671 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:56.666+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:56.717 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:56.709+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:56.762 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:56.757+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:56.806 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:56.800+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:56.807 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:48:56.802+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:48:57.114 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:57.110+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:57.247 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:57.242+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:57.640 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:57.634+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:57.721 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:57.716+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:57.752 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:57.749+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:57.762 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:57.757+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:57.841 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:48:57.835+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:48:58.133 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:58.128+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:58.230 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:58.225+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:58.601 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:58.596+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:58.769 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:58.762+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:58.785 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:58.778+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:58.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:58.767+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:58.828 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:48:58.824+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:48:59.120 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:48:59.115+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:48:59.219 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:48:59.213+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:48:59.605 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:48:59.594+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:48:59.731 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:48:59.726+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:48:59.752 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:48:59.747+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:48:59.775 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:48:59.769+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:48:59.822 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:48:59.816+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:00.144 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:00.139+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:00.267 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:00.261+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:00.555 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:00.549+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:00.709 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:00.697+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:00.765 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:00.759+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:00.784 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:00.776+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:00.785 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:00.782+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:01.147 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:01.142+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:01.243 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:01.237+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:01.528 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:01.523+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:01.664 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:01.658+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:01.746 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:01.741+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:01.770 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:01.765+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:01.773 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:01.765+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:01.773 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:01.768+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:02.188 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:02.183+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:02.284 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:02.278+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:02.499 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:02.486+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:02.665 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:02.659+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:02.757 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:02.752+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:02.778 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:02.773+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:02.796 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:02.790+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:02.799 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:02.794+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:03.233 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:03.227+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:03.335 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:03.324+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:03.504 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:03.498+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:03.702 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:03.697+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:03.741 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:03.736+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:03.764 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:03.759+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:03.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:03.803+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:03.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:03.805+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:04.250 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:04.245+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:04.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:04.313+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:04.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:04.452+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:04.687 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:04.683+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:04.780 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:04.775+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:04.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:04.805+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:04.816 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:04.807+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:04.818 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:04.809+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:05.224 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:05.218+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:05.300 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:05.295+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:05.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:05.452+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:05.718 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:05.712+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:05.734 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:05.730+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:05.767 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:05.763+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:05.844 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:05.837+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:05.845 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:05.840+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:06.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:06.196+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:06.257 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:06.252+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:06.419 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:06.410+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:06.720 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:06.717+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:06.725 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:06.709+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:06.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:06.740+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:06.887 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:06.881+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:06.887 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:06.883+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:07.213 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:07.208+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:07.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:07.222+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:07.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:07.455+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:07.687 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:07.682+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:07.754 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:07.748+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:07.776 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:07.771+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:07.844 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:07.839+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:07.847 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:07.843+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:08.200 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:08.196+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:08.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:08.235+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:08.503 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:08.497+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:08.683 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:08.677+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:08.750 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:08.745+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:08.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:08.750+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:08.817 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:08.810+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:08.817 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:08.813+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:09.159 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:09.155+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:09.235 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:09.231+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:09.541 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:09.533+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:09.675 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:09.670+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:09.735 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:09.728+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:09.761 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:09.754+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:09.866 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:09.860+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:09.867 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:09.862+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:10.115 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:10.110+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:10.274 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:10.270+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:10.509 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:10.503+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:10.633 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:10.629+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:10.722 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:10.716+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:10.734 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:10.729+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:10.898 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:10.892+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:10.899 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:10.894+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:11.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:11.137+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:11.323 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:11.318+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:11.490 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:11.485+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:11.593 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:11.588+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:11.694 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:11.689+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:11.747 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:11.741+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:11.861 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:11.856+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:11.866 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:11.861+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:12.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:12.122+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:12.287 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:12.282+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:12.521 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:12.517+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:12.553 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:12.546+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:12.733 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:12.728+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:12.751 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:12.746+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:12.858 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:12.854+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:12.864 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:12.858+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:13.093 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:13.088+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:13.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:13.314+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:13.527 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:13.521+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:13.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:13.536+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:13.742 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:13.736+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:13.775 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:13.769+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:13.828 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:13.823+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:13.832 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:13.827+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:14.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:14.136+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:14.350 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:14.344+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:14.573 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:14.568+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:14.589 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:14.584+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:14.727 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:14.721+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:14.738 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:14.728+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:14.851 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:14.846+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:14.856 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:14.847+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:15.109 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:15.103+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:15.362 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:49:15.351+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e213 v213) 2019-12-09T18:49:15.377 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:15.371+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:15.599 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:15.595+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:15.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:15.611+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:15.703 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:15.698+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:15.728 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:15.722+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:15.868 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:15.862+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:15.877 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:15.867+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:16.150 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:16.145+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:16.342 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:16.337+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:16.579 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:16.573+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:16.644 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:16.638+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:16.695 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:16.689+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:16.717 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:16.712+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:16.836 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:16.831+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:16.839 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:16.835+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:17.196 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:17.190+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:17.299 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:17.294+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:17.624 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:17.618+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:17.655 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:17.650+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:17.678 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:17.672+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:17.687 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:17.681+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:17.823 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:17.816+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:17.824 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:17.819+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:18.152 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:18.146+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:18.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:18.260+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:18.634 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:18.627+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:18.663 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:18.657+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:18.680 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:18.674+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:18.686 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:18.682+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:18.802 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:18.797+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:18.806 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:18.801+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:19.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:19.163+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:19.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:19.219+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:19.610 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:19.605+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:19.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:19.639+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:19.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:19.721+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:19.737 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:19.731+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:19.779 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:19.774+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:19.783 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:19.779+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:20.152 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:20.147+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:20.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:20.235+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:20.377 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:49:20.370+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e213 v213) 2019-12-09T18:49:20.626 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:20.621+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:20.635 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:20.629+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:20.706 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:20.702+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:20.707 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:20.701+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:20.738 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:20.733+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:20.741 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:20.738+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:21.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:21.124+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:21.270 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:21.265+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:21.630 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:21.625+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:21.671 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:21.667+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:21.672 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:21.669+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:21.697 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:21.690+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:21.768 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:21.763+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:21.768 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:21.765+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:22.181 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:22.174+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:22.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:22.305+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:22.666 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:22.661+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:22.671 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:22.666+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:22.682 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:22.677+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:22.708 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:22.703+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:22.744 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:22.738+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:22.745 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:22.740+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:23.191 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:23.186+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:23.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:23.288+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:23.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:23.622+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:23.684 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:23.678+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:23.704 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:23.698+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:23.724 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:23.720+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:23.770 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:23.764+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:23.772 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:23.766+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:24.229 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:24.222+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:24.322 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:24.316+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:24.647 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:24.641+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:24.654 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:24.645+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:24.672 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:24.666+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:24.724 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:24.719+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:24.742 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:24.736+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:24.743 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:24.738+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:25.243 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:25.239+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:25.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:25.307+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:25.397 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:49:25.392+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e213 v213) 2019-12-09T18:49:25.668 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:25.663+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:25.673 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:25.668+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:25.703 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:25.697+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:25.720 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:25.714+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:25.721 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:25.716+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:25.765 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:25.759+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:26.299 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:26.292+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:26.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:26.303+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:26.665 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:26.658+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:26.692 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:26.686+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:26.693 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:26.688+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:26.702 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:26.696+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:26.705 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:26.700+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:26.783 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:26.778+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:27.328 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:27.323+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:27.332 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:27.325+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:27.666 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:27.661+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:27.710 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:27.704+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:27.712 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:27.706+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:27.731 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:27.725+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:27.733 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:27.727+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:27.750 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:27.745+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:28.292 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:28.287+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:28.353 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:28.347+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:28.699 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:28.692+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:28.701 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:28.696+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:28.702 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:28.697+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:28.721 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:28.712+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:28.722 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:28.715+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:28.800 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:28.796+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:29.285 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:29.279+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:29.333 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:29.323+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:29.666 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:29.660+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:29.689 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:29.683+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:29.713 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:29.707+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:29.714 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:29.709+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:29.753 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:29.747+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:29.834 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:29.829+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:30.282 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:30.277+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:30.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:30.304+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:30.427 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:49:30.417+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e213 v213) 2019-12-09T18:49:30.626 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:30.621+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:30.676 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:30.670+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:30.677 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:30.673+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:30.728 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:30.716+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:30.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:30.785+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:30.875 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:30.869+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:31.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:31.287+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:31.327 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:31.322+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:31.631 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:31.626+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:31.631 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:31.627+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:31.632 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:31.627+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:31.734 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:31.729+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:31.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:31.761+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:31.910 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:31.904+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:32.273 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:32.264+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:32.359 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:32.353+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:32.650 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:32.644+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:32.651 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:32.646+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:32.679 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:32.673+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:32.720 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:32.715+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:32.794 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:32.789+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:32.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:32.894+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:33.247 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:33.240+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:33.399 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:33.393+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:33.612 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:33.606+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:33.613 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:33.608+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:33.684 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:33.678+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:33.684 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:33.679+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:33.807 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:33.802+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:33.932 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:33.926+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:34.256 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:34.250+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:34.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:34.388+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:34.603 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:34.596+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:34.604 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:34.599+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:34.673 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:49:34.666+0000 1694f700 -1 osd.7 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.380832+0000 front 2019-12-09T18:47:59.403871+0000 (oldest deadline 2019-12-09T18:48:43.437404+0000) 2019-12-09T18:49:34.686 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:49:34.680+0000 1694f700 -1 osd.6 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.154316+0000 front 2019-12-09T18:48:00.162954+0000 (oldest deadline 2019-12-09T18:48:43.692360+0000) 2019-12-09T18:49:34.827 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:34.822+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:49:34.943 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:49:34.939+0000 1694f700 -1 osd.2 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:47:59.853646+0000 front 2019-12-09T18:48:00.138552+0000 (oldest deadline 2019-12-09T18:48:44.289045+0000) 2019-12-09T18:49:35.280 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:49:35.277+0000 1694f700 -1 osd.1 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:00.547230+0000 front 2019-12-09T18:48:00.563751+0000 (oldest deadline 2019-12-09T18:48:44.953209+0000) 2019-12-09T18:49:35.394 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:49:35.385+0000 1694f700 -1 osd.5 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.903243+0000 front 2019-12-09T18:48:01.885049+0000 (oldest deadline 2019-12-09T18:48:47.742767+0000) 2019-12-09T18:49:35.502 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:49:35.498+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e213 v213) 2019-12-09T18:49:35.647 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:35.642+0000 1694f700 -1 osd.4 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:48:21.093829+0000 (oldest deadline 2019-12-09T18:49:01.093829+0000) 2019-12-09T18:49:35.648 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:49:35.644+0000 1694f700 -1 osd.4 213 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e211) 2019-12-09T18:49:35.874 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:49:35.863+0000 1694f700 -1 osd.0 213 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:48:01.070251+0000 front 2019-12-09T18:48:01.164637+0000 (oldest deadline 2019-12-09T18:48:43.951630+0000) 2019-12-09T18:50:07.628 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:50:07.586+0000 3578e700 -1 osd.3 219 build_incremental_map_msg missing incremental map 219 2019-12-09T18:50:07.642 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:50:07.613+0000 34f8d700 -1 osd.3 219 build_incremental_map_msg missing incremental map 219 2019-12-09T18:50:07.902 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:50:07.825+0000 31f87700 -1 osd.3 219 build_incremental_map_msg missing incremental map 219 2019-12-09T18:50:08.320 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:50:08.292+0000 37792700 -1 osd.3 219 build_incremental_map_msg missing incremental map 219 2019-12-09T18:50:10.938 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:50:10.925+0000 36f91700 -1 osd.3 219 build_incremental_map_msg missing incremental map 219 2019-12-09T18:50:55.409 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:29:47.258 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.417 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:29:47.267 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.425 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:29:47.249 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.426 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:29:47.251 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.428 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:29:47.253 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.435 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:29:47.259 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.653 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:29:47.478 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.664 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:29:47.514 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:50:55.682 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:29:47.507 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:51:08.244 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:08.238+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:08.580 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:08.575+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:08.616 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:08.610+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:09.288 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:09.282+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:09.407 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:09.403+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:09.576 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:09.572+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:09.591 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:09.585+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:09.597 INFO:teuthology.orchestra.run.smithi174.stderr:testPrefixAndLimit (test.functional.tests.TestContainer) ... ok 2019-12-09T18:51:09.847 INFO:teuthology.orchestra.run.smithi174.stderr:testTooLongName (test.functional.tests.TestContainer) ... ok 2019-12-09T18:51:10.143 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:10.137+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:10.264 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:10.258+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:10.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:10.419+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:10.513 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:10.509+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:10.601 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:10.590+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:10.602 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:10.593+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:11.104 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:11.099+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:11.254 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:11.248+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:11.382 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:11.378+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:11.541 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:11.537+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:11.615 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:11.610+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:11.629 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:11.625+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:12.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:12.140+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:12.288 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:12.281+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:12.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:12.417+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:12.511 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:12.506+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:12.606 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:12.602+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:12.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:12.621+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:13.109 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:13.103+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:13.321 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:13.316+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:13.399 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:13.395+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:13.483 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:13.479+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:13.591 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:13.583+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:13.594 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:13.591+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:14.095 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:14.090+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:14.143 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:14.139+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:14.298 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:14.293+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:14.399 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:14.394+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:14.452 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:14.447+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:14.568 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:14.564+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:14.622 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:14.617+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:15.124 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:15.118+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:15.126 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:15.120+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:15.353 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:15.346+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:15.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:15.419+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:15.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:15.469+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:15.619 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:15.615+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:15.675 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:15.669+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:16.158 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:16.153+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:16.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:16.158+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:16.373 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:16.368+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:16.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:16.414+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:16.475 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:16.471+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:16.575 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:16.571+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:16.716 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:16.711+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:17.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:17.147+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:17.207 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:17.202+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:17.360 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:17.355+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:17.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:17.427+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:17.518 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:17.514+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:17.593 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:17.588+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:17.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:17.699+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:18.179 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:18.173+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:18.242 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:18.236+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:18.368 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:18.363+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:18.450 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:18.446+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:18.546 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:18.541+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:18.618 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:18.613+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:18.721 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:18.717+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:19.225 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:19.220+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:19.277 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:19.271+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:19.332 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:19.326+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:19.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:19.421+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:19.592 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:19.588+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:19.601 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:19.596+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:19.686 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:19.682+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:20.267 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:20.261+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:20.267 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:20.261+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:20.375 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:20.370+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:20.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:20.469+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:20.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:20.556+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:20.584 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:20.579+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:20.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:20.637+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:21.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:21.235+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:21.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:21.261+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:21.401 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:21.397+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:21.495 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:21.490+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:21.585 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:21.576+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:21.605 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:21.602+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:21.695 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:21.690+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:22.225 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:22.220+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:22.266 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:22.261+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:22.388 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:22.383+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:22.548 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:22.543+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:22.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:22.625+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:22.643 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:22.637+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:22.732 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:22.727+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:23.253 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:23.248+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:23.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:23.260+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:23.425 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:23.420+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:23.528 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:23.524+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:23.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:23.620+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:23.683 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:23.678+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:23.718 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:23.714+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:24.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:24.286+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:24.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:24.289+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:24.426 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:24.421+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:24.495 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:24.490+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:24.642 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:24.638+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:24.709 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:24.704+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:24.729 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:24.725+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:25.252 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:25.246+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:25.322 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:25.318+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:25.453 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:25.446+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:25.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:25.463+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:25.665 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:25.660+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:25.675 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:25.671+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:25.739 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:25.735+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:26.282 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:26.278+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:26.368 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:26.362+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:26.440 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:26.434+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:26.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:26.493+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:26.649 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:26.645+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:26.660 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:26.655+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:26.746 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:26.742+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:27.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:27.310+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:27.388 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:27.384+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:27.417 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:27.414+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:27.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:27.493+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:27.643 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:27.639+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:27.682 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:27.678+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:27.759 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:27.755+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:28.319 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:28.315+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:28.419 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:28.413+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:28.465 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:28.461+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:28.524 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:28.516+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:28.644 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:28.636+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:28.658 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:28.654+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:28.733 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:28.728+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:29.332 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:29.326+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:29.432 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:29.427+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:29.437 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:29.433+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:29.553 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:29.549+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:29.635 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:29.628+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:29.676 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:29.671+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:29.697 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:29.690+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:30.311 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:30.306+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:30.441 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:30.437+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:30.480 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:30.476+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:30.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:30.546+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:30.658 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:30.653+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:30.676 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:30.672+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:30.708 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:30.704+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:31.294 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:31.289+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:31.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:31.440+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:31.457 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:31.452+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:31.562 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:31.557+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:31.694 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:31.689+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:31.694 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:31.690+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:31.705 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:31.695+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:32.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:32.308+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:32.422 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:32.416+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:32.482 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:32.477+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:32.581 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:32.575+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:32.670 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:32.666+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:32.672 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:32.668+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:32.694 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:32.691+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:33.280 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:33.276+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:33.379 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:33.375+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:33.476 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:33.471+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:33.575 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:33.571+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:33.627 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:33.623+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:33.661 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:33.656+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:33.739 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:33.734+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:34.238 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:34.234+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:34.366 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:34.360+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:34.499 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:34.494+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:34.614 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:34.610+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:34.659 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:34.653+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:34.716 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:34.710+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:34.773 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:34.769+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:35.200 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:35.195+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:35.386 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:35.382+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:35.537 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:35.530+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:35.583 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:35.579+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:35.637 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:35.632+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:35.745 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:35.740+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:35.788 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:35.784+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:36.252 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:36.247+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:36.429 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:36.423+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:36.546 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:36.538+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:36.613 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:36.607+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:36.636 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:36.629+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:36.747 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:36.741+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:36.759 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:36.753+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:37.299 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:37.293+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:37.434 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:37.429+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:37.570 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:37.558+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:37.708 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:37.571+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:37.709 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:37.668+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:37.733 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:37.727+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:37.801 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:37.794+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:38.316 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:38.308+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:38.464 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:38.460+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:38.555 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:38.550+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:38.565 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:38.560+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:38.653 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:38.648+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:38.697 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:38.692+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:38.820 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:38.815+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:39.288 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:39.281+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:39.426 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:39.421+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:39.549 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:39.543+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:39.617 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:39.611+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:39.706 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:39.700+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:39.735 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:39.729+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:39.864 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:39.858+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:40.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:40.334+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:40.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:40.455+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:40.523 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:40.516+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:40.599 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:40.594+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:40.676 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:40.670+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:40.720 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:40.714+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:40.892 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:40.887+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:41.270 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:51:41.266+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e219 v219) 2019-12-09T18:51:41.308 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:41.304+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:41.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:41.488+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:41.536 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:41.530+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:41.576 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:41.572+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:41.643 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:41.638+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:41.751 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:41.746+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:41.880 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:41.875+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:42.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:42.273+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:42.544 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:42.539+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:42.554 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:42.541+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:42.600 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:42.595+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:42.610 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:42.604+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:42.748 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:42.742+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:42.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:42.908+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:43.315 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:43.311+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:43.517 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:43.512+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:43.577 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:43.572+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:43.586 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:43.581+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:43.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:43.588+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:43.770 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:43.764+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:43.933 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:43.927+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:44.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:44.315+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:44.537 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:44.532+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:44.554 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:44.549+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:44.617 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:44.611+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:44.622 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:44.617+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:44.739 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:44.732+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:44.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:44.924+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:45.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:45.349+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:45.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:51:45.545+0000 1694f700 -1 osd.3 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.275685+0000 front 2019-12-09T18:50:25.279116+0000 (oldest deadline 2019-12-09T18:51:09.321557+0000) 2019-12-09T18:51:45.562 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:51:45.556+0000 1694f700 -1 osd.7 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.510361+0000 front 2019-12-09T18:50:27.670710+0000 (oldest deadline 2019-12-09T18:51:13.311712+0000) 2019-12-09T18:51:45.592 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:45.586+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:51:45.638 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:51:45.633+0000 1694f700 -1 osd.1 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.376758+0000 front 2019-12-09T18:50:27.412270+0000 (oldest deadline 2019-12-09T18:51:10.364370+0000) 2019-12-09T18:51:45.748 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:51:45.743+0000 1694f700 -1 osd.0 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.742979+0000 front 2019-12-09T18:50:26.081813+0000 (oldest deadline 2019-12-09T18:51:08.256229+0000) 2019-12-09T18:51:45.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:51:45.959+0000 1694f700 -1 osd.2 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:27.777552+0000 front 2019-12-09T18:50:27.781173+0000 (oldest deadline 2019-12-09T18:51:07.787388+0000) 2019-12-09T18:51:46.317 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:51:46.313+0000 1694f700 -1 osd.6 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.052920+0000 front 2019-12-09T18:50:24.875501+0000 (oldest deadline 2019-12-09T18:51:09.788496+0000) 2019-12-09T18:51:46.361 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:51:46.351+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e219 v219) 2019-12-09T18:51:46.545 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:51:46.540+0000 1694f700 -1 osd.5 219 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:50:25.372857+0000 front 2019-12-09T18:50:25.374133+0000 (oldest deadline 2019-12-09T18:51:08.077198+0000) 2019-12-09T18:52:19.112 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:19.105+0000 31f87700 -1 osd.2 223 build_incremental_map_msg missing incremental map 223 2019-12-09T18:52:19.190 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:19.152+0000 32788700 -1 osd.2 223 build_incremental_map_msg missing incremental map 223 2019-12-09T18:52:19.303 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:19.259+0000 3638f700 -1 osd.2 223 build_incremental_map_msg missing incremental map 223 2019-12-09T18:52:21.434 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:21.431+0000 3538d700 -1 osd.2 225 build_incremental_map_msg missing incremental map 223 2019-12-09T18:52:44.619 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:44.614+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:45.652 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:45.648+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:45.916 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:45.912+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:46.654 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:46.650+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:46.814 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:46.807+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:46.960 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:46.957+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:47.441 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:47.434+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:47.664 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:47.659+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:47.829 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:47.823+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:47.875 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:47.872+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:48.012 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:48.009+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:48.358 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:48.349+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:48.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:48.399+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:48.654 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:48.650+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:48.817 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:48.812+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:48.904 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:48.900+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:49.040 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:49.036+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:49.334 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:49.328+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:49.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:49.439+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:49.666 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:49.661+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:49.831 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:49.825+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:49.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:49.948+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:49.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:49.993+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:50.333 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:50.328+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:50.501 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:50.494+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:50.665 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:50.657+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:50.802 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:50.797+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:50.912 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:50.907+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:51.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:51.002+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:51.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:51.289+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:51.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:51.468+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:51.647 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:51.642+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:51.848 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:51.842+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:51.925 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:51.919+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:52.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:52.020+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:52.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:52.301+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:52.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:52.461+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:52.636 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:52.629+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:52.865 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:52.860+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:52.922 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:52.916+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:52.980 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:52.977+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:53.039 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:53.034+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:53.298 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:53.294+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:53.438 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:53.435+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:53.679 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:53.674+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:53.871 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:53.866+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:53.953 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:53.947+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:53.996 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:53.991+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:54.046 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:54.040+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:54.317 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:54.309+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:54.420 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:54.414+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:54.710 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:54.704+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:54.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:54.884+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:54.957 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:54.950+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:55.027 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:55.022+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:55.068 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:55.062+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:55.286 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:55.280+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:55.385 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:55.381+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:55.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:55.735+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:55.877 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:55.872+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:55.910 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:55.905+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:56.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:56.045+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:56.072 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:56.068+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:56.306 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:56.301+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:56.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:56.408+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:56.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:56.771+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:56.855 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:56.849+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:56.913 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:56.906+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:57.034 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:57.029+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:57.084 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:57.069+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:57.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:57.300+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:57.453 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:57.447+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:57.771 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:57.764+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:57.872 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:57.867+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:57.887 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:57.881+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:57.993 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:57.987+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:52:57.994 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:57.989+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:58.055 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:58.049+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:58.287 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:58.281+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:58.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:58.452+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:58.825 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:58.819+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:58.880 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:58.874+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:52:58.888 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:58.883+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:59.018 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:59.011+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:52:59.019 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:52:59.014+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:52:59.035 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:52:59.029+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:52:59.329 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:52:59.323+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:52:59.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:52:59.443+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:52:59.834 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:52:59.830+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:52:59.862 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:52:59.856+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:52:59.934 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:52:59.928+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:00.056 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:00.050+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:00.062 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:00.055+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:00.063 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:00.058+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:00.352 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:00.347+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:00.489 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:00.483+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:00.809 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:00.803+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:00.896 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:00.887+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:00.901 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:00.895+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:01.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:01.026+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:01.073 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:01.065+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:01.076 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:01.068+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:01.356 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:01.350+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:01.488 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:01.482+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:01.771 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:01.766+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:01.858 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:01.852+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:01.917 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:01.912+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:02.001 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:01.995+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:02.046 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:02.039+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:02.047 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:02.042+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:02.338 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:02.329+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:02.505 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:02.500+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:02.759 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:02.754+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:02.916 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:02.908+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:02.937 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:02.932+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:03.039 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:03.034+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:03.072 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:03.066+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:03.076 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:03.071+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:03.389 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:03.383+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:03.524 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:03.519+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:03.737 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:03.730+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:03.878 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:03.873+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:03.969 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:03.965+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:04.038 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:04.033+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:04.041 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:04.037+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:04.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:04.046+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:04.418 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:04.412+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:04.504 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:04.498+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:04.709 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:04.703+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:04.916 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:04.911+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:04.994 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:04.988+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:05.027 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:05.021+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:05.028 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:05.024+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:05.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:05.088+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:05.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:05.455+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:05.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:05.469+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:05.757 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:05.751+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:05.895 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:05.889+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:06.025 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:06.018+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:06.026 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:06.021+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:06.026 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:06.021+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:06.102 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:06.096+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:06.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:06.462+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:06.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:06.465+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:06.764 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:06.758+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:06.911 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:06.905+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:07.015 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:07.010+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:07.060 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:07.054+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:07.060 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:07.056+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:07.113 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:07.108+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:07.425 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:07.421+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:07.440 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:07.434+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:07.798 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:07.792+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:07.941 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:07.935+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:07.983 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:07.976+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:08.117 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:08.111+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:08.118 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:08.113+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:08.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:08.136+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:08.433 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:08.427+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:08.438 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:08.432+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:08.782 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:08.777+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:08.990 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:08.984+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:08.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:08.991+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:09.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:09.067+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:09.075 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:09.069+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:09.194 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:09.186+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:09.425 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:09.419+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:09.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:09.456+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:09.756 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:09.751+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:09.959 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:09.954+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:09.969 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:09.965+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:10.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:10.057+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:10.065 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:10.060+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:10.219 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:10.214+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:10.424 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:10.418+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:10.450 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:10.445+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:10.717 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:10.712+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:10.919 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:10.912+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:10.936 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:10.931+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:11.022 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:11.016+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:11.023 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:11.018+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:11.223 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:11.218+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:11.472 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:11.465+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:11.496 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:11.490+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:11.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:11.721+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:11.876 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:11.871+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:11.969 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:11.964+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:12.066 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:12.060+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:12.067 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:12.062+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:12.214 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:12.209+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:12.438 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:12.434+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:12.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:12.461+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:12.682 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:12.677+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:12.896 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:12.891+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:12.981 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:12.975+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:13.068 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:13.062+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:13.069 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:13.064+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:13.255 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:13.251+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:13.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:13.406+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:13.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:13.441+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:13.692 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:13.686+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:13.914 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:13.910+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:14.033 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:14.028+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:14.053 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:14.046+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:14.054 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:14.048+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:14.216 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:14.211+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:14.375 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:14.369+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:14.474 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:14.469+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:14.671 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:14.667+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:14.926 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:14.922+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:15.000 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:14.994+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:15.088 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:15.082+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:15.089 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:15.084+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:15.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:15.166+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:15.334 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:15.329+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:15.499 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:15.493+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:15.657 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:15.653+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:15.965 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:15.960+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:15.978 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:15.973+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:16.110 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:16.097+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:16.112 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:16.107+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:16.203 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:16.198+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:16.338 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:16.334+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:16.472 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:16.465+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:16.689 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:16.685+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:16.929 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:16.922+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:16.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:16.943+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:17.128 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:17.121+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:17.129 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:17.124+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:17.216 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:17.211+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:17.296 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:17.291+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:17.468 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:17.463+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:17.700 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:17.696+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:17.905 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:17.902+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:17.939 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:17.933+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:18.122 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:18.115+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:18.123 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:18.118+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:18.250 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:18.237+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:18.263 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:18.257+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:18.436 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:18.422+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:18.736 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:18.733+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:18.935 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:18.930+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:18.981 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:53:18.976+0000 1694f700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.966399+0000 front 2019-12-09T18:52:00.967661+0000 (oldest deadline 2019-12-09T18:52:46.081924+0000) 2019-12-09T18:53:19.078 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:19.071+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:19.081 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:19.073+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:19.210 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:19.203+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:19.294 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:19.288+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:19.452 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:19.448+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:19.790 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:19.784+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:19.925 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:19.915+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:20.053 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:20.047+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:20.057 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:20.052+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:20.242 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:20.236+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:20.292 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:20.285+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:20.458 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:20.453+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:20.790 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:20.786+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:20.929 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:20.925+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:21.041 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:21.036+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:21.045 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:21.040+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:21.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:21.198+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:21.333 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:21.329+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:21.456 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:21.451+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:21.737 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:53:21.733+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e225 v225) 2019-12-09T18:53:21.794 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:21.790+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:21.943 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:21.939+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:22.002 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:21.994+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:22.003 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:21.998+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:22.167 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:22.163+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:22.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:22.313+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:22.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:22.407+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:22.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:22.767+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:22.927 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:22.923+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:23.014 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:23.007+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:23.015 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:23.010+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:23.196 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:23.192+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:23.330 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:23.325+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:23.386 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:23.380+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:23.747 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:23.743+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:23.958 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:23.953+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:24.036 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:24.029+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:24.038 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:24.032+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:24.178 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:24.169+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:24.344 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:24.339+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:24.360 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:24.351+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:24.733 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:24.729+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:24.969 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:24.965+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:25.075 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:25.068+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:25.076 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:25.071+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:25.183 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:25.178+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:25.341 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:25.335+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:25.362 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:25.355+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:25.784 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:25.779+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:25.992 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:25.988+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:26.068 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:26.062+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:26.069 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:26.064+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:26.184 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:26.179+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:26.310 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:26.305+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:26.349 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:26.344+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:26.762 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:53:26.755+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e225 v225) 2019-12-09T18:53:26.816 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:26.812+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:26.972 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:26.968+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:27.075 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:27.066+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:27.077 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:27.069+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:27.228 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:27.225+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:27.322 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:27.317+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:27.400 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:27.393+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:27.816 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:27.812+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:27.987 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:27.983+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:28.066 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:28.059+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:28.070 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:28.062+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:28.222 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:28.216+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:28.309 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:28.304+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:28.358 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:28.352+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:28.819 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:28.815+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:28.984 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:28.980+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:29.031 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:29.024+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:29.032 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:29.027+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:29.185 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:29.181+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:29.328 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:29.322+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:29.371 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:29.366+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:29.853 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:29.847+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:29.963 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:29.959+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:30.070 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:30.064+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:30.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:30.069+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:30.177 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:30.173+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:30.364 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:30.358+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:30.416 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:30.410+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:30.849 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:30.844+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:30.954 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:30.950+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:31.106 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:31.100+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:31.107 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:31.102+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:31.180 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:53:31.175+0000 1694f700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.625375+0000 front 2019-12-09T18:52:01.570283+0000 (oldest deadline 2019-12-09T18:52:45.659411+0000) 2019-12-09T18:53:31.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:53:31.407+0000 1694f700 -1 osd.6 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.840012+0000 front 2019-12-09T18:52:00.730013+0000 (oldest deadline 2019-12-09T18:52:46.564417+0000) 2019-12-09T18:53:31.455 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:53:31.449+0000 1694f700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:02.355557+0000 front 2019-12-09T18:52:02.283601+0000 (oldest deadline 2019-12-09T18:52:48.144495+0000) 2019-12-09T18:53:31.809 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:53:31.803+0000 1694f700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:00.636772+0000 front 2019-12-09T18:52:00.568967+0000 (oldest deadline 2019-12-09T18:52:44.005533+0000) 2019-12-09T18:53:31.857 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:53:31.848+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e225 v225) 2019-12-09T18:53:31.963 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:53:31.958+0000 1694f700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:52:01.325353+0000 front 2019-12-09T18:52:01.384763+0000 (oldest deadline 2019-12-09T18:52:47.176741+0000) 2019-12-09T18:53:32.069 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:32.065+0000 1694f700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:52:17.300461+0000 (oldest deadline 2019-12-09T18:52:57.300461+0000) 2019-12-09T18:53:32.070 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:53:32.067+0000 1694f700 -1 osd.4 225 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e225) 2019-12-09T18:53:42.259 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:34.109 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.273 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:34.123 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.337 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:34.162 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.338 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:34.163 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.344 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:34.169 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.346 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:34.170 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.508 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:34.333 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.510 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:34.335 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:42.570 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:34.419 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.458 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:49.308 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.490 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:49.340 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.506 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:49.331 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.521 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:32:49.371 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.557 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:49.362 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.557 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:49.369 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.558 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:49.374 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.578 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:32:49.403 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:53:57.579 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:32:49.403 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:07.741 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.494+0000 35f8f700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:07.745 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.500+0000 30f85700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:07.746 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.502+0000 35f8f700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:07.778 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.600+0000 34f8d700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:07.786 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.615+0000 35f8f700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:07.923 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.681+0000 34f8d700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:08.031 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:07.917+0000 32f89700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:08.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:08.268+0000 37f93700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:08.348 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:54:08.297+0000 32788700 -1 osd.3 231 build_incremental_map_msg missing incremental map 231 2019-12-09T18:54:53.803 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:33:45.653 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.817 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:33:45.641 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.819 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:33:45.644 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.820 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:33:45.645 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.824 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:33:45.674 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.828 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:33:45.653 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.918 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:33:45.743 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.921 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:33:45.746 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:54:53.956 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:33:45.806 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:55:04.357 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:04.352+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:05.004 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:04.999+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:05.324 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:05.319+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:05.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:05.994+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:06.014 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:06.009+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:06.288 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:06.283+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:06.978 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:06.973+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:07.016 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:07.011+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:07.256 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:07.251+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:07.503 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:07.499+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:07.938 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:07.933+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:07.964 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:07.959+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:08.002 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:07.997+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:08.311 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:08.306+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:08.478 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:08.472+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:08.925 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:08.919+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:09.010 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:09.004+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:09.017 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:09.012+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:09.206 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:09.200+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:09.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:09.350+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:09.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:09.467+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:09.515 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:09.509+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:09.945 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:09.941+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:09.970 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:09.967+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:10.026 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:10.022+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:10.179 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:10.175+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:10.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:10.308+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:10.490 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:10.485+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:10.520 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:10.516+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:10.921 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:10.915+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:10.966 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:10.960+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:11.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:11.054+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:11.182 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:11.177+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:11.321 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:11.317+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:11.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:11.440+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:11.558 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:11.554+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:11.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:11.900+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:11.980 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:11.976+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:12.066 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:12.060+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:12.151 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:12.146+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:12.363 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:12.358+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:12.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:12.431+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:12.562 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:12.557+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:12.859 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:12.855+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:13.023 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:13.018+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:13.097 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:13.092+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:13.184 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:13.181+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:13.386 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:13.382+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:13.424 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:13.418+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:13.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:13.565+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:13.892 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:13.887+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:14.021 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:14.017+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:14.140 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:14.135+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:14.193 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:14.190+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:14.346 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:14.337+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:14.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:14.421+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:14.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:14.545+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:14.877 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:14.872+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:15.066 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:15.058+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:15.184 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:15.179+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:15.241 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:15.232+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:15.367 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:15.362+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:15.430 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:15.424+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:15.596 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:15.590+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:15.873 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:15.867+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:16.038 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:16.032+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:16.153 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:16.147+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:16.225 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:16.214+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:16.344 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:16.338+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:16.393 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:16.387+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:16.554 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:16.547+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:16.827 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:16.822+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:16.994 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:16.989+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:17.193 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:17.188+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:17.195 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:17.190+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:17.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:17.343+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:17.432 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:17.427+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:17.541 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:17.537+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:17.849 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:17.844+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:17.999 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:17.992+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:18.152 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:18.145+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:18.232 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:18.226+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:18.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:18.309+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:18.414 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:18.408+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:18.585 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:18.579+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:18.829 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:18.824+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:18.957 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:18.952+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:19.154 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:19.149+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:19.214 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:19.210+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:19.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:19.305+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:19.436 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:19.431+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:19.614 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:19.608+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:19.786 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:19.781+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:19.933 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:19.927+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:20.133 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:20.128+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:20.251 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:20.246+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:20.302 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:20.297+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:20.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:20.468+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:20.661 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:20.656+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:20.791 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:20.785+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:20.977 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:20.971+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:21.182 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:21.176+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:21.285 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:21.280+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:21.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:21.310+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:21.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:21.431+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:21.629 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:21.623+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:21.770 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:21.765+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:21.968 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:21.963+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:22.223 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:22.217+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:22.298 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:22.293+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:22.337 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:22.331+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:22.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:22.416+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:22.645 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:22.640+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:22.775 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:22.770+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:22.963 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:22.958+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:23.259 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:23.252+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:23.283 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:23.277+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:23.296 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:23.291+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:23.475 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:23.460+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:23.667 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:23.661+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:23.763 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:23.758+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:23.924 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:23.919+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:24.253 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:24.247+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:24.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:24.256+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:24.323 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:24.319+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:24.448 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:24.441+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:24.657 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:24.650+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:24.747 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:24.742+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:24.896 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:24.890+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:25.223 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:25.217+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:25.246 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:25.241+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:25.309 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:25.304+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:25.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:25.424+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:25.707 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:25.701+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:25.783 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:25.776+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:25.937 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:25.931+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:26.209 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:26.203+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:26.213 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:26.208+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:26.285 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:26.279+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:26.467 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:26.460+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:26.723 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:26.718+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:26.828 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:26.823+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:26.903 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:26.898+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:27.252 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:27.246+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:27.253 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:27.247+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:27.261 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:27.255+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:27.421 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:27.417+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:27.709 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:27.700+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:27.842 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:27.836+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:27.904 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:27.898+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:28.254 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:28.249+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:28.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:28.278+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:28.309 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:28.304+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:28.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:28.431+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:28.752 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:28.747+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:28.821 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:28.816+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:28.905 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:28.901+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:29.227 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:29.220+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:29.297 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:29.293+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:29.329 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:29.324+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:29.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:29.419+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:29.716 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:29.710+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:29.818 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:29.812+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:29.861 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:29.856+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:30.238 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:30.234+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:30.289 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:30.283+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:30.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:30.290+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:30.438 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:30.432+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:30.737 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:30.733+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:30.830 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:30.825+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:30.881 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:30.870+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:31.231 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:31.226+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:31.274 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:31.270+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:31.318 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:31.313+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:31.397 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:31.391+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:31.700 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:31.694+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:31.801 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:31.793+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:31.899 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:31.893+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:32.265 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:32.260+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:32.272 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:32.268+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:32.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:32.308+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:32.416 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:32.411+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:32.657 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:32.651+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:32.752 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:32.748+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:32.877 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:32.872+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:33.231 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:33.227+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:33.307 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:33.302+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:33.326 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:33.321+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:33.468 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:33.464+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:33.626 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:33.621+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:33.726 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:33.721+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:33.884 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:33.877+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:34.205 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:34.199+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:34.302 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:34.297+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:34.306 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:34.301+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:34.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:34.469+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:34.668 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:34.663+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:34.712 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:34.707+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:34.849 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:34.844+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:35.245 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:35.240+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:35.309 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:35.303+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:35.341 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:35.337+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:35.488 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:35.484+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:35.697 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:35.692+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:35.720 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:35.714+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:35.868 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:35.863+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:36.252 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:36.247+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:36.298 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:36.291+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:36.324 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:36.319+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:36.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:36.467+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:36.700 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:36.694+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:36.727 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:36.722+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:36.917 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:36.911+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:37.289 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:37.284+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:37.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:37.289+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:37.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:37.342+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:37.521 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:37.514+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:37.722 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:37.718+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:37.780 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:37.775+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:37.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:37.942+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:38.272 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:38.266+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:38.310 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:38.304+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:38.379 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:38.373+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:38.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:38.504+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:38.744 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:38.738+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:38.788 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:38.783+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:38.950 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:38.945+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:39.265 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:39.260+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:39.318 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:39.314+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:39.400 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:39.393+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:39.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:39.491+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:39.729 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:39.724+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:39.793 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:39.786+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:39.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:39.957+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:40.229 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:40.222+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:40.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:40.279+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:40.453 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:40.448+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:40.510 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:40.506+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:40.782 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:40.776+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:40.809 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:40.803+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:40.918 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:40.914+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:41.240 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:41.235+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:41.263 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:41.257+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:41.469 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:41.466+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:41.525 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:41.521+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:41.762 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:41.758+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:41.775 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:41.771+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:41.929 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:41.922+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:55:42.252 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:55:42.248+0000 1694f700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.869409+0000 front 2019-12-09T18:54:24.011435+0000 (oldest deadline 2019-12-09T18:55:08.562892+0000) 2019-12-09T18:55:42.293 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:55:42.289+0000 1694f700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:21.955641+0000 front 2019-12-09T18:54:22.340887+0000 (oldest deadline 2019-12-09T18:55:07.003049+0000) 2019-12-09T18:55:42.428 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:55:42.424+0000 1694f700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:20.608904+0000 front 2019-12-09T18:54:25.063434+0000 (oldest deadline 2019-12-09T18:55:04.046316+0000) 2019-12-09T18:55:42.568 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:55:42.563+0000 1694f700 -1 osd.3 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.116515+0000 front 2019-12-09T18:54:24.927865+0000 (oldest deadline 2019-12-09T18:55:09.320101+0000) 2019-12-09T18:55:42.748 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:55:42.743+0000 1694f700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:25.114632+0000 front 2019-12-09T18:54:24.517043+0000 (oldest deadline 2019-12-09T18:55:07.333441+0000) 2019-12-09T18:55:42.749 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:55:42.742+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e231 v231) 2019-12-09T18:55:42.793 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:55:42.789+0000 1694f700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:24.797594+0000 front 2019-12-09T18:54:21.410784+0000 (oldest deadline 2019-12-09T18:55:04.655300+0000) 2019-12-09T18:55:42.945 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:55:42.942+0000 1694f700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:54:23.324494+0000 front 2019-12-09T18:54:24.894084+0000 (oldest deadline 2019-12-09T18:55:05.784933+0000) 2019-12-09T18:56:19.202 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.169+0000 32788700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:19.206 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.180+0000 32788700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:19.532 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.498+0000 33f8b700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:19.668 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.648+0000 31f87700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:19.735 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.690+0000 3378a700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:20.035 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:19.967+0000 31786700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:20.256 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:20.211+0000 36f91700 -1 osd.4 236 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:21.577 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:21.538+0000 32788700 -1 osd.4 237 build_incremental_map_msg missing incremental map 236 2019-12-09T18:56:41.880 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:41.875+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:42.006 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:42.000+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:42.148 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:42.143+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:42.890 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:42.886+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:42.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:42.918+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:43.033 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:43.028+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:43.106 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:43.101+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:43.886 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:43.880+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:43.934 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:43.927+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:44.060 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:44.054+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:44.118 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:44.113+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:44.588 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:44.581+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:44.905 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:44.899+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:44.908 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:44.902+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:45.085 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:45.080+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:45.173 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:45.167+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:45.545 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:45.540+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:45.728 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:45.723+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:45.893 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:45.887+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:45.937 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:45.929+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:46.054 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:46.048+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:46.220 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:46.215+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:46.523 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:46.518+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:46.735 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:46.729+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:46.870 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:46.863+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:46.943 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:46.937+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:47.048 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:47.043+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:47.234 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:47.229+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:47.486 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:47.481+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:47.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:47.767+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:47.865 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:47.861+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:47.932 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:47.928+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:48.020 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:48.014+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:48.213 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:48.207+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:48.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:48.447+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:48.645 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:48.588+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:48.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:48.739+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:48.913 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:48.907+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:48.930 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:48.919+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:49.043 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:49.038+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:49.220 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:49.214+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:49.449 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:49.445+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:49.608 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:49.604+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:49.706 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:49.700+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:49.884 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:49.879+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:49.929 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:49.923+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:50.072 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:50.065+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:50.224 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:50.219+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:50.502 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:50.496+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:50.653 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:50.648+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:50.715 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:50.708+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:50.893 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:50.889+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:50.909 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:50.904+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:51.052 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:51.046+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:51.272 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:51.268+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:51.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:51.454+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:51.625 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:51.620+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:51.762 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:51.756+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:51.867 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:51.862+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:51.907 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:51.902+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:52.038 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:52.032+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:52.247 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:52.241+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:52.415 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:52.410+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:52.634 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:52.629+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:52.731 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:52.727+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:52.860 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:52.854+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:52.885 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:52.879+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:53.042 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:53.035+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:53.258 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:53.253+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:53.377 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:53.371+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:53.663 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:53.658+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:53.719 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:53.715+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:53.910 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:53.905+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:53.925 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:53.921+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:54.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:54.036+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:54.314 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:54.308+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:54.365 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:54.358+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:54.719 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:54.709+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:54.765 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:54.760+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:54.898 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:54.893+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:54.952 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:54.948+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:55.012 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:55.007+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:55.280 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:55.274+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:55.356 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:55.351+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:55.675 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:55.670+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:55.807 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:55.802+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:55.879 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:55.871+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:55.982 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:55.978+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:56.028 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:56.022+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:56.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:56.279+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:56.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:56.392+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:56.638 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:56.632+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:56:56.639 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:56.634+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:56.828 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:56.824+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:56.894 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:56.889+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:56.947 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:56.942+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:57.075 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:57.069+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:57.259 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:57.252+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:57.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:57.387+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:57.666 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:57.661+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:56:57.667 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:57.662+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:57.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:57.846+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:57.936 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:57.928+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:57.975 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:57.971+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:58.049 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:58.043+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:58.279 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:58.274+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:58.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:58.348+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:58.669 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:58.661+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:56:58.670 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:58.665+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:58.879 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:58.875+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:58.950 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:58.946+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:56:58.982 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:56:58.977+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:56:59.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:59.001+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:59.282 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:56:59.274+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:56:59.326 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:56:59.320+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:56:59.630 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:59.621+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:56:59.630 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:56:59.626+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:56:59.855 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:56:59.849+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:56:59.989 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:56:59.983+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:56:59.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:56:59.994+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:00.005 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:00.001+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:00.247 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:00.242+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:00.293 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:00.288+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:00.633 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:00.627+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:00.634 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:00.629+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:00.892 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:00.887+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:01.003 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:00.997+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:01.044 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:01.040+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:01.044 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:01.039+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:01.226 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:01.221+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:01.305 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:01.300+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:01.620 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:01.615+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:01.624 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:01.620+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:01.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:01.921+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:02.018 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:02.012+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:02.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:02.057+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:02.093 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:02.085+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:02.186 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:02.181+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:02.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:02.290+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:02.666 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:02.660+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:02.668 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:02.662+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:02.910 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:02.901+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:03.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:03.002+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:03.081 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:03.077+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:03.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:03.133+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:03.189 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:03.184+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:03.292 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:03.287+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:03.648 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:03.642+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:03.649 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:03.644+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:03.961 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:03.955+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:03.963 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:03.957+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:04.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:04.082+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:04.156 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:04.152+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:04.193 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:04.188+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:04.285 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:04.279+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:04.611 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:04.605+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:04.612 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:04.608+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:04.932 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:04.927+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:04.967 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:04.963+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:05.059 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:05.055+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:05.146 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:05.142+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:05.240 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:05.234+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:05.338 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:05.333+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:05.567 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:05.561+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:05.569 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:05.564+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:05.933 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:05.927+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:05.950 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:05.946+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:06.098 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:06.092+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:06.157 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:06.153+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:06.207 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:06.203+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:06.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:06.298+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:06.601 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:06.597+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:06.605 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:06.602+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:06.932 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:06.928+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:06.966 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:06.963+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:07.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:07.129+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:07.175 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:07.171+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:07.182 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:07.178+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:07.318 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:07.314+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:07.649 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:07.644+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:07.650 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:07.646+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:07.934 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:07.930+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:07.996 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:07.993+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:08.130 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:08.126+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:08.154 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:08.151+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:08.180 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:08.176+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:08.359 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:08.354+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:08.686 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:08.681+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:08.687 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:08.683+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:08.939 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:08.934+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:09.026 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:09.023+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:09.111 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:09.106+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:09.126 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:09.121+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:09.199 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:09.193+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:09.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:09.389+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:09.659 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:09.652+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:09.660 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:09.655+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:09.933 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:09.928+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:10.043 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:10.039+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:10.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:10.110+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:10.161 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:10.157+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:10.229 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:10.223+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:10.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:10.400+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:10.694 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:10.687+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:10.695 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:10.690+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:10.995 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:10.988+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:11.055 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:11.050+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:11.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:11.086+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:11.124 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:11.114+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:11.194 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:11.189+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:11.441 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:11.435+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:11.733 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:11.727+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:11.734 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:11.730+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:12.007 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:12.003+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:12.058 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:12.053+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:12.071 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:12.066+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:12.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:12.081+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:12.232 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:12.226+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:12.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:12.438+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:12.772 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:12.765+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:12.773 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:12.768+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:13.023 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:13.017+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:13.046 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:13.042+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:13.054 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:13.050+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:13.070 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:13.066+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:13.227 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:13.222+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:13.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:13.456+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:13.818 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:13.811+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:13.819 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:13.814+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:14.006 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:14.001+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:14.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:14.035+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:14.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:14.090+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:14.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:14.100+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:14.211 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:14.206+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:14.437 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:14.431+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:14.830 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:14.825+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:14.834 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:14.830+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:15.058 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:15.050+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:15.066 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:15.061+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:15.111 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:15.107+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:15.120 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:15.115+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:15.180 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:15.175+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:15.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:15.438+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:15.851 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:15.846+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:15.854 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:15.850+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:16.020 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:16.015+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:16.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:16.105+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:16.129 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:16.124+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:16.164 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:16.159+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:16.190 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:16.185+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:16.406 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:16.399+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:16.862 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:16.857+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:16.866 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:16.862+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:17.067 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:17.063+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:17.140 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:17.135+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:17.160 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:17.154+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:17.174 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:17.169+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:17.216 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:17.211+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:17.412 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:17.407+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:17.870 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:17.863+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:17.871 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:17.866+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:18.029 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:18.024+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:18.033 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:57:18.028+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e237 v237) 2019-12-09T18:57:18.133 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:18.128+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:18.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:18.155+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:18.190 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:18.185+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:18.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:18.223+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:18.368 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:18.362+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:18.923 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:18.915+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:18.925 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:18.920+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:19.007 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:19.000+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:19.164 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:19.158+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:19.177 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:19.172+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:19.202 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:19.198+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:19.224 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:19.219+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:19.385 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:19.378+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:19.976 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:19.970+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:19.977 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:19.972+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:20.049 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:20.045+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:20.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:20.133+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:20.171 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:20.167+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:20.209 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:20.203+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:20.257 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:20.251+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:20.357 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:20.351+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:20.937 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:20.931+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:20.938 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:20.933+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:21.019 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:21.015+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:21.135 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:21.130+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:21.193 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:21.189+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:21.193 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:21.190+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:21.223 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:21.216+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:21.400 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:21.395+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:21.963 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:21.957+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:21.964 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:21.959+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:21.995 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:21.991+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:22.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:22.128+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:22.184 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:22.180+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:22.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:22.232+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:22.246 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:22.240+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:22.384 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:22.378+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:22.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:22.976+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:22.997 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:22.990+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:23.003 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:22.997+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:23.053 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:57:23.046+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e237 v237) 2019-12-09T18:57:23.172 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:23.167+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:23.194 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:23.190+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:23.233 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:23.229+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:23.235 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:23.232+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:23.350 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:23.345+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:23.977 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:23.974+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:23.999 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:23.993+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:24.001 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:23.995+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:24.164 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:24.160+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:24.215 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:24.210+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:24.217 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:24.212+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:24.224 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:24.219+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:24.323 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:24.317+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:24.992 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:24.984+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:25.021 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:25.016+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:25.022 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:25.018+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:25.174 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:25.168+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:25.206 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:25.201+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:25.262 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:25.209+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:25.279 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:25.273+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:25.305 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:25.299+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:25.983 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:25.971+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:25.983 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:25.974+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:26.010 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:26.006+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:26.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:26.138+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:26.183 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:26.179+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:26.219 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:26.214+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:26.236 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:26.231+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:26.262 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:26.257+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:26.990 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:26.984+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:26.991 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:26.986+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:27.002 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:26.997+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:27.099 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:27.094+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:27.176 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:27.172+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:27.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:27.216+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:27.229 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:27.225+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:27.304 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:27.298+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:28.015 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:28.011+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:28.031 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:28.026+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:28.035 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:28.031+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:28.072 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:57:28.066+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e237 v237) 2019-12-09T18:57:28.135 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:28.130+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:28.186 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:28.181+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:28.187 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:28.183+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:28.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:28.197+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:28.261 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:28.255+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:28.975 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:28.971+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:29.025 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:29.019+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:29.027 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:29.022+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:29.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:29.137+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:29.161 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:29.156+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:29.187 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:29.182+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:29.228 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:29.217+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:29.290 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:29.285+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:30.001 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:29.992+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:30.002 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:29.997+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:30.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:30.012+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:30.146 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:30.141+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:30.147 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:30.142+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:30.181 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:30.175+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:30.268 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:30.263+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:30.338 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:30.332+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:31.003 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:30.999+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:31.038 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:31.032+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:31.039 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:31.034+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:31.124 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:31.119+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:31.157 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:31.151+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:31.211 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:31.205+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:31.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:31.269+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:31.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:31.298+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:31.999 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:31.994+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:32.064 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:32.058+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:32.065 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:32.060+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:32.143 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:32.138+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:32.156 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:32.151+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:32.196 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:32.191+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:57:32.277 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:57:32.273+0000 1694f700 -1 osd.0 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:56.839700+0000 front 2019-12-09T18:55:56.831360+0000 (oldest deadline 2019-12-09T18:56:42.458599+0000) 2019-12-09T18:57:32.327 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:57:32.321+0000 1694f700 -1 osd.6 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:59.936130+0000 front 2019-12-09T18:55:59.875286+0000 (oldest deadline 2019-12-09T18:56:43.891249+0000) 2019-12-09T18:57:33.003 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:57:32.996+0000 1694f700 -1 osd.2 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.029684+0000 front 2019-12-09T18:56:00.032823+0000 (oldest deadline 2019-12-09T18:56:45.288574+0000) 2019-12-09T18:57:33.032 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:33.027+0000 1694f700 -1 osd.4 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T18:56:15.670871+0000 (oldest deadline 2019-12-09T18:56:55.670871+0000) 2019-12-09T18:57:33.036 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T18:57:33.032+0000 1694f700 -1 osd.4 237 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:114759 7.4 7.7a216314 (undecoded) ondisk+retry+write+known_if_redirected e235) 2019-12-09T18:57:33.140 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:57:33.135+0000 1694f700 -1 osd.7 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:55:55.426516+0000 front 2019-12-09T18:55:55.893952+0000 (oldest deadline 2019-12-09T18:56:41.232676+0000) 2019-12-09T18:57:33.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:57:33.138+0000 1694f700 -1 osd.1 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.417719+0000 front 2019-12-09T18:56:00.418862+0000 (oldest deadline 2019-12-09T18:56:41.374895+0000) 2019-12-09T18:57:33.144 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:57:33.140+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 43sec e237 v237) 2019-12-09T18:57:33.210 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:57:33.205+0000 1694f700 -1 osd.5 237 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:56:00.350176+0000 front 2019-12-09T18:56:00.347165+0000 (oldest deadline 2019-12-09T18:56:41.371439+0000) 2019-12-09T18:58:00.938 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:00.866+0000 34f8d700 -1 osd.3 242 build_incremental_map_msg missing incremental map 242 2019-12-09T18:58:01.220 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:01.212+0000 38794700 -1 osd.3 242 build_incremental_map_msg missing incremental map 242 2019-12-09T18:58:01.854 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:01.816+0000 11caf700 -1 osd.3 243 build_incremental_map_msg missing incremental map 243 2019-12-09T18:58:01.878 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:01.853+0000 34f8d700 -1 osd.3 243 build_incremental_map_msg missing incremental map 243 2019-12-09T18:58:02.215 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:02.209+0000 114ae700 -1 osd.3 243 build_incremental_map_msg missing incremental map 243 2019-12-09T18:58:02.373 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:02.355+0000 34f8d700 -1 osd.3 243 build_incremental_map_msg missing incremental map 243 2019-12-09T18:58:04.024 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:04.011+0000 33f8b700 -1 osd.3 244 build_incremental_map_msg missing incremental map 242 2019-12-09T18:58:04.025 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:58:04.011+0000 33f8b700 -1 osd.3 244 build_incremental_map_msg missing incremental map 243 2019-12-09T18:58:53.805 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:37:45.654 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.811 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:37:45.661 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.828 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:37:45.652 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.832 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:37:45.657 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.834 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:37:45.659 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:53.839 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:37:45.664 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:54.073 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:37:45.897 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:54.086 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:37:45.910 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T18:58:54.101 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:37:45.951 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T18:59:01.352 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:01.345+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:02.383 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:02.377+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:02.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:02.492+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:03.170 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:03.165+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:03.347 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:03.343+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:03.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:03.503+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:03.512 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:03.506+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:03.516 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:03.511+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:04.141 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:04.134+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:04.344 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:04.339+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:04.476 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:04.469+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:04.509 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:04.503+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:04.519 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:04.513+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:05.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:05.107+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:05.268 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:05.262+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:05.393 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:05.388+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:05.484 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:05.479+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:05.549 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:05.544+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:05.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:05.549+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:05.649 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:05.643+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:06.088 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:06.084+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:06.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:06.307+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:06.399 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:06.394+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:06.470 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:06.464+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:06.529 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:06.523+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:06.574 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:06.570+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:06.626 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:06.622+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:07.078 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:07.066+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:07.288 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:07.281+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:07.432 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:07.427+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:07.444 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:07.439+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:07.504 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:07.499+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:07.578 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:07.573+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:07.628 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:07.623+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:08.047 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:08.041+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:08.256 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:08.251+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:08.465 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:08.459+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:08.472 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:08.467+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:08.505 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:08.497+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:08.536 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:08.532+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:08.644 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:08.639+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:09.094 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:09.089+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:09.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:09.296+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:09.469 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:09.463+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:09.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:09.463+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:09.473 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:09.466+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:09.565 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:09.560+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:09.686 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:09.681+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:10.093 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:10.088+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:10.318 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:10.313+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:10.477 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:10.466+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:10.500 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:10.495+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:10.517 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:10.511+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:10.596 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:10.591+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:10.734 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:10.729+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:11.090 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:11.085+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:11.365 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:11.359+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:11.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:11.457+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:11.476 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:11.470+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:11.480 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:11.475+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:11.567 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:11.563+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:11.772 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:11.766+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:12.096 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:12.090+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:12.350 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:12.345+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:12.463 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:12.457+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:12.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:12.469+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:12.482 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:12.478+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:12.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:12.615+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:12.824 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:12.818+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:13.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:13.122+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:13.342 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:13.337+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:13.457 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:13.453+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:13.461 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:13.456+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:13.532 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:13.526+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:13.643 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:13.639+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:13.828 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:13.823+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:14.135 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:14.131+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:14.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:14.297+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:14.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:14.442+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:14.482 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:14.477+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:14.582 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:14.576+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:14.635 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:14.631+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:14.870 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:14.865+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:15.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:15.122+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:15.325 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:15.320+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:15.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:15.455+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:15.469 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:15.465+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:15.565 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:15.560+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:15.625 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:15.620+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:15.899 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:15.893+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:16.163 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:16.159+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:16.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:16.285+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:16.438 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:16.433+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:16.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:16.465+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:16.584 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:16.580+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:16.606 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:16.600+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:16.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:16.918+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:17.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:17.147+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:17.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:17.293+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:17.406 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:17.400+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:17.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:17.493+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:17.576 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:17.572+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:17.622 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:17.617+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:17.912 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:17.907+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:18.136 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:18.131+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:18.282 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:18.277+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:18.371 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:18.366+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:18.483 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:18.478+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:18.542 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:18.537+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:18.621 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:18.616+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:18.941 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:18.934+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:19.164 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:19.157+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:19.250 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:19.246+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:19.394 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:19.390+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:19.441 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:19.436+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:19.585 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:19.581+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:19.613 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:19.609+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:19.914 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:19.910+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:20.187 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:20.182+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:20.274 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:20.268+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:20.372 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:20.367+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:20.435 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:20.430+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:20.570 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:20.566+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:20.659 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:20.655+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:20.895 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:20.890+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:21.216 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:21.211+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:21.283 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:21.279+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:21.345 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:21.341+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:21.424 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:21.418+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:21.555 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:21.551+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:21.650 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:21.646+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:21.903 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:21.899+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:22.185 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:22.180+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:22.338 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:22.334+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:22.395 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:22.389+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:22.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:22.466+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:22.581 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:22.576+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:22.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:22.615+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:22.899 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:22.894+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:23.160 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:23.156+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:23.339 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:23.335+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:23.357 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:23.351+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:23.458 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:23.455+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:23.589 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:23.585+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:23.671 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:23.666+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:23.934 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:23.930+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:24.184 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:24.179+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:24.316 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:24.311+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:24.316 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:24.311+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:24.461 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:24.455+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:24.611 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:24.606+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:24.648 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:24.642+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:24.930 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:24.926+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:25.169 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:25.163+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:25.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:25.307+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:25.343 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:25.339+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:25.491 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:25.486+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:25.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:25.572+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:25.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:25.614+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:25.965 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:25.960+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:26.173 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:26.168+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:26.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:26.297+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:26.303 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:26.299+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:26.450 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:26.446+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:26.587 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:26.581+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:26.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:26.624+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:26.957 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:26.952+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:27.176 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:27.171+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:27.263 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:27.258+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:27.351 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:27.346+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:27.469 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:27.464+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:27.587 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:27.582+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:27.665 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:27.661+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:27.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:27.994+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:28.149 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:28.144+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:28.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:28.287+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:28.400 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:28.395+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:28.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:28.469+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:28.640 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:28.635+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:28.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:28.643+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:29.032 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:29.028+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:29.157 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:29.152+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:29.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:29.274+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:29.418 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:29.412+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:29.477 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:29.472+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:29.610 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:29.605+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:29.613 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:29.607+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:30.065 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:30.060+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:30.119 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:30.114+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:30.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:30.309+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:30.433 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:30.428+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:30.465 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:30.460+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:30.566 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:30.561+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:30.609 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:30.596+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:31.112 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:31.104+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:31.135 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:31.130+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:31.341 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:31.336+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:31.390 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:31.385+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:31.496 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:31.492+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:31.570 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:31.566+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:31.608 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:31.604+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:32.081 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:32.076+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:32.142 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:32.137+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:32.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:32.297+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:32.361 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:32.349+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:32.457 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:32.453+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:32.604 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:32.599+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:32.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:32.616+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:33.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:33.105+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:33.130 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:33.124+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:33.307 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:33.304+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:33.338 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:33.332+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:33.500 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:33.495+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:33.587 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:59:33.583+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e244 v244) 2019-12-09T18:59:33.589 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:33.584+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:33.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:33.640+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:34.148 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:34.143+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:34.163 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:34.157+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:34.292 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:34.287+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:34.319 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:34.314+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:34.457 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:34.453+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:34.621 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:34.607+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:34.668 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:34.663+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:35.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:35.120+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:35.161 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:35.155+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:35.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:35.277+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:35.321 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:35.316+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:35.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:35.440+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:35.621 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:35.615+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:35.661 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:35.657+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:36.168 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:36.163+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:36.194 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:36.190+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:36.252 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:36.247+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:36.347 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:36.341+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:36.461 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:36.457+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:36.597 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:36.590+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:36.635 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:36.630+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:37.158 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:37.153+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:37.237 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:37.233+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:37.285 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:37.280+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:37.336 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:37.330+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:37.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:37.468+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:37.639 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:37.633+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:37.639 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:37.634+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T18:59:38.184 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T18:59:38.179+0000 1694f700 -1 osd.5 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:19.543394+0000 front 2019-12-09T18:58:20.377093+0000 (oldest deadline 2019-12-09T18:59:03.030736+0000) 2019-12-09T18:59:38.193 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T18:59:38.188+0000 1694f700 -1 osd.1 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.354488+0000 front 2019-12-09T18:58:20.356163+0000 (oldest deadline 2019-12-09T18:59:05.347870+0000) 2019-12-09T18:59:38.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T18:59:38.291+0000 1694f700 -1 osd.2 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:20.517104+0000 front 2019-12-09T18:58:19.403128+0000 (oldest deadline 2019-12-09T18:59:04.844731+0000) 2019-12-09T18:59:38.355 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T18:59:38.350+0000 1694f700 -1 osd.7 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.330261+0000 front 2019-12-09T18:58:18.150942+0000 (oldest deadline 2019-12-09T18:59:01.330887+0000) 2019-12-09T18:59:38.452 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T18:59:38.447+0000 1694f700 -1 osd.3 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.235299+0000 front 2019-12-09T18:58:18.152076+0000 (oldest deadline 2019-12-09T18:59:02.201702+0000) 2019-12-09T18:59:38.647 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T18:59:38.642+0000 1694f700 -1 osd.6 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.174987+0000 front 2019-12-09T18:58:18.238561+0000 (oldest deadline 2019-12-09T18:59:02.899468+0000) 2019-12-09T18:59:38.652 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T18:59:38.648+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e244 v244) 2019-12-09T18:59:38.685 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T18:59:38.677+0000 1694f700 -1 osd.0 244 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T18:58:18.837831+0000 front 2019-12-09T18:58:18.961946+0000 (oldest deadline 2019-12-09T18:59:03.288337+0000) 2019-12-09T19:00:13.911 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:00:13.731+0000 3378a700 -1 osd.3 248 build_incremental_map_msg missing incremental map 248 2019-12-09T19:00:14.164 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:00:13.857+0000 36790700 -1 osd.3 248 build_incremental_map_msg missing incremental map 248 2019-12-09T19:00:14.230 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:00:14.090+0000 37792700 -1 osd.3 248 build_incremental_map_msg missing incremental map 248 2019-12-09T19:00:14.422 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:00:14.351+0000 33f8b700 -1 osd.3 248 build_incremental_map_msg missing incremental map 248 2019-12-09T19:00:15.166 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:15.099+0000 36f91700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:15.224 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:15.110+0000 31786700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:15.442 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:15.419+0000 31f87700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:15.473 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:15.447+0000 30f85700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:15.556 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:15.532+0000 36f91700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:16.828 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:16.817+0000 32f89700 -1 osd.4 249 build_incremental_map_msg missing incremental map 249 2019-12-09T19:00:36.754 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:36.750+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:37.344 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:37.339+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:37.436 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:37.431+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:37.758 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:37.754+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:38.374 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:38.369+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:38.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:38.488+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:38.789 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:38.784+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:39.391 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:39.386+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:39.546 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:39.540+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:39.728 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:39.724+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:39.819 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:39.816+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:39.938 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:39.933+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:40.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:40.392+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:40.598 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:40.590+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:40.707 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:40.700+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:40.859 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:40.855+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:40.912 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:40.908+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:41.379 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:41.373+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:41.629 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:41.613+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:41.734 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:41.728+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:41.750 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:41.745+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:41.907 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:41.903+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:41.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:41.919+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:42.351 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:42.345+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:42.646 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:42.640+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:42.743 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:42.738+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:42.762 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:42.749+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:42.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:42.867+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:42.915 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:42.910+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:43.396 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:43.390+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:43.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:43.599+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:43.777 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:43.771+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:43.792 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:43.787+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:43.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:43.899+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:43.951 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:43.945+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:44.385 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:44.380+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:44.606 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:44.601+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:44.628 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:44.620+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:44.780 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:44.773+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:44.818 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:44.815+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:44.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:44.905+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:45.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:44.995+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:45.430 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:45.424+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:45.568 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:45.562+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:45.609 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:45.605+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:45.787 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:45.783+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:45.825 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:45.819+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:45.905 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:45.898+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:46.034 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:46.029+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:46.477 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:46.472+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:46.607 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:46.600+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:46.610 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:46.604+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:46.818 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:46.813+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:46.841 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:46.836+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:46.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:46.857+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:47.069 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:47.064+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:47.492 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:47.487+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:47.606 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:47.602+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:47.645 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:47.640+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:47.853 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:47.847+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:47.867 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:47.861+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:47.871 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:47.867+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:48.063 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:48.058+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:48.508 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:48.503+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:48.571 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:48.566+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:48.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:48.643+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:48.829 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:48.824+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:48.893 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:48.888+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:48.925 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:48.921+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:49.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:49.025+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:49.520 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:49.515+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:49.544 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:49.538+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:49.699 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:49.694+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:49.842 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:49.838+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:49.858 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:49.853+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:49.962 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:49.957+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:50.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:49.994+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:50.514 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:50.508+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:50.565 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:50.561+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:50.745 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:50.739+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:50.843 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:50.838+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:50.882 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:50.877+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:51.012 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:51.005+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:51.013 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:51.009+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:51.530 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:51.524+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:51.531 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:51.526+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:51.559 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:51.553+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:51.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:51.780+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:51.823 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:51.820+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:51.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:51.833+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:51.992 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:51.988+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:52.015 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:52.011+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:52.522 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:52.516+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:52.562 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:52.556+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:52.563 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:52.558+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:52.829 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:52.822+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:52.846 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:52.841+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:52.886 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:52.880+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:52.954 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:52.950+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:53.057 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:53.052+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:53.478 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:53.473+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:53.572 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:53.565+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:53.574 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:53.568+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:53.851 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:53.845+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:53.856 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:53.853+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:53.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:53.870+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:53.942 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:53.938+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:54.047 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:54.042+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:54.511 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:54.506+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:54.552 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:54.542+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:54.552 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:54.544+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:54.840 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:54.835+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:54.849 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:54.844+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:54.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:54.847+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:54.917 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:54.911+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:55.045 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:55.040+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:55.531 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:55.525+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:55.595 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:55.589+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:55.596 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:55.591+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:55.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:55.805+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:55.864 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:55.859+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:55.893 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:55.888+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:55.932 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:55.927+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:56.052 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:56.047+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:56.516 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:56.510+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:56.581 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:56.576+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:56.585 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:56.581+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:56.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:56.787+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:56.878 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:56.873+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:56.908 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:56.902+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:56.953 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:56.949+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:57.021 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:57.016+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:57.502 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:57.496+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:57.624 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:57.617+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:57.624 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:57.620+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:57.817 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:57.812+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:57.839 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:57.835+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:57.943 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:57.938+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:57.993 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:57.990+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:57.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:57.994+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:58.521 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:58.515+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:58.631 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:58.626+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:58.635 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:58.631+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:58.870 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:58.865+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:58.886 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:58.881+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:58.984 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:58.980+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:58.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:58.987+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:00:59.010 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:00:59.006+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:00:59.561 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:00:59.555+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:00:59.594 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:59.589+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:00:59.598 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:00:59.594+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:00:59.895 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:00:59.889+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:00:59.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:00:59.926+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:00:59.985 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:00:59.980+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:00:59.987 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:00:59.983+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:00.047 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:00.042+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:00.586 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:00.580+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:00.589 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:00.585+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:00.596 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:00.589+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:00.922 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:00.917+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:00.974 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:00.970+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:00.983 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:00.978+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:01.009 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:01.005+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:01.046 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:01.043+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:01.565 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:01.559+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:01.639 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:01.632+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:01.640 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:01.635+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:01.888 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:01.883+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:01.952 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:01.946+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:02.004 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:01.999+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:02.020 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:02.008+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:02.020 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:02.010+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:02.595 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:02.590+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:02.686 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:02.677+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:02.687 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:02.679+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:02.856 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:02.851+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:02.988 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:02.982+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:02.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:02.993+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:03.040 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:03.037+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:03.041 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:03.036+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:03.566 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:03.561+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:03.707 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:03.701+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:03.708 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:03.703+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:03.914 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:03.908+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:03.951 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:03.947+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:03.973 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:03.968+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:04.007 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:04.003+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:04.018 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:04.010+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:04.529 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:04.523+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:04.684 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:04.679+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:04.684 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:04.681+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:04.886 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:04.881+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:04.928 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:04.923+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:04.938 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:04.933+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:04.978 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:04.974+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:05.042 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:05.039+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:05.558 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:05.553+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:05.704 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:05.699+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:05.705 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:05.701+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:05.898 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:05.893+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:05.941 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:05.935+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:05.947 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:05.939+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:05.953 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:05.946+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:06.019 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:06.014+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:06.601 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:06.595+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:06.691 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:06.686+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:06.697 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:06.692+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:06.869 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:06.858+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:06.917 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:06.911+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:06.959 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:06.955+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:06.973 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:06.968+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:06.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:06.985+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:07.564 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:07.558+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:07.725 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:07.719+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:07.727 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:07.721+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:07.882 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:07.877+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:07.896 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:07.892+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:07.953 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:07.949+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:07.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:07.956+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:08.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:07.997+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:08.595 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:08.589+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:08.744 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:08.737+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:08.745 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:08.740+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:08.888 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:08.882+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:08.918 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:08.911+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:08.925 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:08.921+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:08.959 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:01:08.952+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e249 v249) 2019-12-09T19:01:09.003 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:08.997+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:09.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:09.023+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:09.575 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:09.570+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:09.727 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:09.721+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:09.728 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:09.723+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:09.873 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:09.861+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:09.893 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:09.887+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:09.943 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:09.939+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:10.012 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:10.005+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:10.037 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:10.032+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:10.592 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:10.586+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:10.775 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:10.769+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:10.833 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:10.771+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:10.856 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:10.850+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:10.877 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:10.871+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:10.960 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:10.955+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:11.010 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:11.006+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:11.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:11.014+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:11.616 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:11.611+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:11.819 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:11.814+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:11.823 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:11.819+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:11.879 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:11.874+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:11.908 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:11.903+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:11.933 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:11.929+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:12.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:12.011+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:12.056 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:12.047+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:12.627 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:12.622+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:12.857 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:12.850+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:12.858 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:12.853+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:12.867 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:12.861+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:12.896 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:12.890+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:12.967 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:12.963+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:12.972 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:12.967+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:13.060 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:13.057+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:13.670 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:13.666+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:13.875 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:13.870+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:13.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:13.881+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:13.893 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:13.887+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:13.926 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:13.921+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:13.982 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:01:13.974+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e249 v249) 2019-12-09T19:01:14.006 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:14.001+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:14.010 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:14.006+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:14.021 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:14.014+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:14.722 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:14.716+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:14.847 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:14.841+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:14.909 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:14.903+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:14.910 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:14.905+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:14.980 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:14.974+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:14.990 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:14.985+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:15.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:15.002+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:15.021 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:15.017+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:15.758 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:15.752+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:15.868 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:15.863+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:15.905 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:15.900+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:15.906 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:15.902+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:16.011 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:16.006+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:16.016 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:16.011+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:16.032 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:16.026+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:16.044 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:16.037+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:16.721 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:16.715+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:16.873 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:16.865+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:16.873 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:16.867+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:16.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:16.886+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:16.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:16.972+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:17.010 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:17.005+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:17.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:17.050+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:17.057 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:17.049+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:17.745 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:17.740+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:17.872 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:17.862+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:17.872 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:17.865+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:17.895 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:17.890+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:17.950 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:17.947+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:18.037 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:18.032+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:18.076 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:18.072+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:18.103 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:18.097+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:18.719 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:18.714+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:18.878 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:18.873+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:18.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:18.878+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:18.891 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:18.885+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:18.936 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:18.932+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:19.001 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:01:18.992+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e249 v249) 2019-12-09T19:01:19.021 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:19.016+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:19.048 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:19.043+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:19.134 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:19.125+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:19.711 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:19.707+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:19.876 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:19.871+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:19.928 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:19.921+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:19.932 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:19.924+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:19.933 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:19.928+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:19.982 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:19.977+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:20.054 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:20.049+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:20.157 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:20.153+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:20.721 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:20.717+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:20.879 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:20.874+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:20.901 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:20.896+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:20.906 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:20.901+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:20.940 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:20.935+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:20.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:20.969+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:21.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:21.024+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:21.188 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:21.184+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:21.706 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:21.703+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:21.873 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:21.869+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:21.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:21.879+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:21.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:21.881+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:21.988 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:21.982+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:21.991 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:21.983+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:22.028 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:22.023+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:22.226 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:22.216+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:22.684 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:22.679+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:22.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:22.841+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:22.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:22.844+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:22.912 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:22.907+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:22.980 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:22.974+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:22.987 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:22.982+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:22.991 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:22.985+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:23.196 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:23.190+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:23.715 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:01:23.709+0000 1694f700 -1 osd.6 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.621874+0000 front 2019-12-09T18:59:53.384861+0000 (oldest deadline 2019-12-09T19:00:37.108249+0000) 2019-12-09T19:01:23.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:23.846+0000 1694f700 -1 osd.4 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:00:11.151380+0000 (oldest deadline 2019-12-09T19:00:51.151380+0000) 2019-12-09T19:01:23.853 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:23.848+0000 1694f700 -1 osd.4 249 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e248) 2019-12-09T19:01:23.870 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:01:23.864+0000 1694f700 -1 osd.7 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.667392+0000 front 2019-12-09T18:59:54.979025+0000 (oldest deadline 2019-12-09T19:00:36.665345+0000) 2019-12-09T19:01:23.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:01:23.955+0000 1694f700 -1 osd.0 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.548514+0000 front 2019-12-09T18:59:55.581567+0000 (oldest deadline 2019-12-09T19:00:40.789010+0000) 2019-12-09T19:01:23.990 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:01:23.985+0000 1694f700 -1 osd.1 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:55.734452+0000 front 2019-12-09T18:59:55.686685+0000 (oldest deadline 2019-12-09T19:00:39.644428+0000) 2019-12-09T19:01:24.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:01:24.017+0000 1694f700 -1 osd.2 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:53.711616+0000 front 2019-12-09T18:59:53.712274+0000 (oldest deadline 2019-12-09T19:00:35.975017+0000) 2019-12-09T19:01:24.095 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:01:24.083+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e249 v249) 2019-12-09T19:01:24.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:01:24.178+0000 1694f700 -1 osd.5 249 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T18:59:54.136535+0000 front 2019-12-09T18:59:54.122393+0000 (oldest deadline 2019-12-09T19:00:38.800604+0000) 2019-12-09T19:01:24.798 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:24.615+0000 36f91700 -1 osd.4 250 build_incremental_map_msg missing incremental map 249 2019-12-09T19:01:50.561 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:50.554+0000 36f91700 -1 osd.3 249 build_incremental_map_msg missing incremental map 248 2019-12-09T19:01:54.232 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.215+0000 34f8d700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:54.373 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.282+0000 37792700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:54.531 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.345+0000 3578e700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:54.678 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.518+0000 31f87700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:54.738 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.675+0000 11caf700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:54.843 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:01:54.839+0000 36f91700 -1 osd.4 253 build_incremental_map_msg missing incremental map 253 2019-12-09T19:01:56.673 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:56.665+0000 3578e700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:56.815 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:56.794+0000 32f89700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:56.877 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:56.796+0000 32f89700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:57.065 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:56.858+0000 3578e700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:57.234 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:57.080+0000 35f8f700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:57.279 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:57.148+0000 3578e700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:01:57.507 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:01:57.403+0000 38794700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:02:00.126 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:02:00.016+0000 33f8b700 -1 osd.3 254 build_incremental_map_msg missing incremental map 254 2019-12-09T19:02:19.749 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:11.599 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.754 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:11.604 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.820 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:11.645 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.829 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:11.654 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.830 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:11.655 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.833 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:11.658 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:19.929 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:11.779 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:20.032 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:11.857 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:20.093 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:11.918 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.906 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:43.756 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.915 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:43.766 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.929 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:43.753 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.933 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:43.758 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.934 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:43.759 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.939 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:43.764 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:51.976 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:41:43.826 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:52.003 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:41:43.828 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:52.005 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:41:43.830 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:02:56.050 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:02:56.046+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:02:56.503 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:02:56.497+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:02:56.777 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:02:56.773+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:02:57.047 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:02:57.042+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:02:57.218 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:02:57.212+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:02:57.537 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:02:57.532+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:02:57.681 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:02:57.676+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:02:57.812 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:02:57.806+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:02:58.070 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:02:58.067+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:02:58.253 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:02:58.249+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:02:58.527 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:02:58.522+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:02:58.641 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:02:58.636+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:02:58.743 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:02:58.738+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:02:58.772 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:02:58.768+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:02:59.072 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:02:59.067+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:02:59.216 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:02:59.210+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:02:59.532 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:02:59.525+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:02:59.694 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:02:59.689+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:02:59.778 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:02:59.774+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:02:59.783 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:02:59.779+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:00.080 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:00.076+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:00.214 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:00.209+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:00.580 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:00.570+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:00.677 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:00.673+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:00.777 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:00.772+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:00.828 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:00.824+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:01.100 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:01.096+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:01.220 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:01.216+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:01.620 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:01.616+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:01.668 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:01.662+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:01.780 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:01.775+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:01.878 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:01.873+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:02.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:02.135+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:02.204 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:02.199+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:02.528 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:02.523+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:02.654 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:02.649+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:02.664 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:02.659+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:02.802 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:02.794+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:02.853 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:02.848+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:03.100 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:03.096+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:03.215 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:03.210+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:03.495 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:03.489+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:03.627 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:03.622+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:03.693 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:03.690+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:03.798 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:03.793+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:03.901 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:03.897+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:04.087 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:04.082+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:04.264 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:04.259+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:04.503 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:04.498+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:04.597 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:04.592+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:04.690 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:04.686+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:04.850 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:04.845+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:04.910 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:04.905+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:05.072 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:05.067+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:05.242 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:05.238+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:05.530 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:05.525+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:05.625 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:05.621+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:05.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:05.673+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:05.883 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:05.879+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:05.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:05.944+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:06.099 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:06.094+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:06.270 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:06.266+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:06.524 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:06.518+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:06.649 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:06.644+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:06.716 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:06.712+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:06.899 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:06.894+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:06.993 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:06.988+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:07.053 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:07.048+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:07.281 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:07.277+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:07.558 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:07.552+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:07.609 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:07.604+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:07.715 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:07.712+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:07.920 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:07.915+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:07.985 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:07.980+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:08.059 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:08.054+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:08.259 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:08.253+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:08.511 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:08.506+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:08.605 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:08.594+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:08.710 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:08.704+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:08.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:08.926+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:08.952 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:08.946+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:09.108 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:09.102+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:09.305 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:09.301+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:09.492 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:09.485+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:09.636 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:09.632+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:09.757 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:09.752+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:09.965 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:09.960+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:09.974 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:09.970+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:10.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:10.127+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:10.285 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:10.280+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:10.536 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:10.530+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:10.634 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:10.628+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:10.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:10.722+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:10.973 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:10.966+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:10.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:10.964+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:11.092 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:11.087+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:11.305 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:11.300+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:11.593 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:11.582+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:11.680 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:11.676+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:11.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:11.734+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:11.980 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:11.974+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:11.984 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:11.980+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:12.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:12.126+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:12.320 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:12.315+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:12.632 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:12.628+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:12.735 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:12.730+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:12.738 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:12.733+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:13.002 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:12.995+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:13.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:13.009+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:13.113 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:13.109+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:13.363 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:13.359+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:13.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:13.605+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:13.720 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:13.715+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:13.752 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:13.747+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:13.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:13.969+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:14.039 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:14.034+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:14.076 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:14.070+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:14.332 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:14.327+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:14.609 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:14.604+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:14.684 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:14.678+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:14.789 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:14.783+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:14.972 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:14.967+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:15.024 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:15.019+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:15.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:15.081+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:15.329 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:15.325+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:15.581 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:15.575+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:15.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:15.666+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:15.835 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:15.830+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:15.943 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:15.939+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:15.991 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:15.985+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:16.052 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:16.048+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:16.384 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:16.379+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:16.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:16.598+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:16.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:16.628+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:16.863 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:16.854+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:16.980 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:16.975+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:17.041 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:17.037+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:17.095 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:17.091+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:17.421 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:17.416+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:17.576 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:17.569+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:17.609 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:17.605+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:17.840 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:17.836+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:18.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:18.004+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:18.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:18.068+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:18.121 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:18.116+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:18.433 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:18.429+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:18.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:18.545+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:18.597 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:18.593+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:18.847 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:18.842+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:19.006 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:19.001+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:19.088 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:19.083+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:19.112 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:19.108+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:19.493 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:19.488+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:19.540 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:19.535+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:19.629 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:19.624+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:19.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:19.804+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:20.039 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:20.035+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:20.078 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:20.074+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:20.086 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:20.081+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:20.517 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:20.510+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:20.517 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:20.507+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:20.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:20.672+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:20.802 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:20.797+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:21.058 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:21.054+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:21.062 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:21.057+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:21.073 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:21.068+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:21.468 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:21.463+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:21.517 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:21.511+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:21.651 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:21.646+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:21.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:21.790+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:22.064 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:22.059+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:22.083 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:22.077+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:22.117 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:22.112+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:22.463 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:22.458+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:22.507 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:22.501+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:22.648 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:22.643+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:22.847 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:22.842+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:23.037 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:23.032+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:23.063 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:23.059+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:23.105 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:23.098+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:23.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:23.470+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:23.510 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:23.503+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:23.696 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:23.692+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:23.814 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:23.809+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:24.076 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:24.072+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:24.084 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:24.079+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:24.092 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:24.082+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:24.475 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:24.469+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:24.539 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:24.533+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:24.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:24.710+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:24.812 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:24.807+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:25.036 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:25.032+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:25.066 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:25.061+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:25.079 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:25.075+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:25.463 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:25.457+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:25.538 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:25.533+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:25.682 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:25.678+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:25.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:25.805+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:26.030 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:26.025+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:26.101 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:26.096+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:26.117 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:26.112+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:26.484 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:26.479+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:26.551 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:26.547+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:26.727 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:26.724+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:26.776 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:26.768+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:27.070 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:27.064+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:27.096 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:27.092+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:27.149 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:27.142+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:27.456 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:27.451+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:27.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:27.600+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:27.733 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:27.728+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:27.760 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:27.757+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:28.100 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:28.096+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:28.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:28.098+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:28.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:28.125+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:28.505 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:28.501+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:28.622 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:28.616+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:28.709 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:28.703+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:28.728 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:28.724+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:29.092 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:29.086+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:29.105 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:29.102+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:29.152 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:29.147+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:29.559 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:29.555+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:29.637 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:29.633+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:29.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:29.736+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:29.760 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:29.755+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:29.772 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:03:29.751+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e254 v254) 2019-12-09T19:03:30.102 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:30.093+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:30.118 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:30.113+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:30.153 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:30.148+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:30.540 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:30.535+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:30.672 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:30.666+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:30.722 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:30.718+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:30.773 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:30.768+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:31.110 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:31.105+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:31.135 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:31.131+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:31.185 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:31.179+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:31.564 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:31.559+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:31.694 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:31.689+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:31.725 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:31.719+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:31.767 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:31.761+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:32.072 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:32.068+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:32.147 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:32.143+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:32.212 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:32.208+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:32.591 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:32.586+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:32.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:32.706+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:32.733 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:32.724+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:32.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:32.806+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:33.123 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:33.118+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:33.192 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:33.187+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:33.221 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:33.216+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:33.627 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:33.623+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:33.663 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:33.659+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:33.726 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:33.720+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:33.846 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:33.841+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:03:34.130 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:03:34.126+0000 1694f700 -1 osd.0 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:15.076661+0000 front 2019-12-09T19:02:15.513279+0000 (oldest deadline 2019-12-09T19:02:57.795057+0000) 2019-12-09T19:03:34.208 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:03:34.203+0000 1694f700 -1 osd.6 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.611987+0000 front 2019-12-09T19:02:13.611411+0000 (oldest deadline 2019-12-09T19:02:56.267513+0000) 2019-12-09T19:03:34.238 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:03:34.233+0000 1694f700 -1 osd.1 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:16.445771+0000 front 2019-12-09T19:02:13.802344+0000 (oldest deadline 2019-12-09T19:02:55.991856+0000) 2019-12-09T19:03:34.608 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:03:34.605+0000 1694f700 -1 osd.3 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:13.631146+0000 front 2019-12-09T19:02:13.627979+0000 (oldest deadline 2019-12-09T19:02:57.083865+0000) 2019-12-09T19:03:34.682 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:03:34.677+0000 1694f700 -1 osd.2 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:14.809612+0000 front 2019-12-09T19:02:14.535939+0000 (oldest deadline 2019-12-09T19:02:57.416413+0000) 2019-12-09T19:03:34.723 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:03:34.718+0000 1694f700 -1 osd.7 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:12.975593+0000 front 2019-12-09T19:02:13.540785+0000 (oldest deadline 2019-12-09T19:03:02.106790+0000) 2019-12-09T19:03:34.834 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:03:34.829+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e254 v254) 2019-12-09T19:03:34.885 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:03:34.881+0000 1694f700 -1 osd.5 254 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:02:11.883744+0000 front 2019-12-09T19:02:11.874257+0000 (oldest deadline 2019-12-09T19:02:55.787273+0000) 2019-12-09T19:04:04.296 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:04.255+0000 21452700 -1 reset not still connected to 0xed1c380 2019-12-09T19:04:04.343 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:04.255+0000 21452700 -1 reset not still connected to 0x158aee30 2019-12-09T19:04:07.383 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:07.335+0000 34f8d700 -1 osd.0 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:07.556 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:07.419+0000 32788700 -1 osd.0 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:07.574 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:07.463+0000 31786700 -1 osd.0 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:07.604 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:07.588+0000 31f87700 -1 osd.6 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:07.828 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:07.772+0000 36f91700 -1 osd.0 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:07.932 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:07.919+0000 3478c700 -1 osd.0 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.019 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:07.923+0000 37f93700 -1 osd.6 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.057 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:07.975+0000 37f93700 -1 osd.6 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.109 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:08.074+0000 34f8d700 -1 osd.6 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.117 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:04:08.040+0000 36790700 -1 osd.3 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.621 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:04:08.362+0000 32788700 -1 osd.3 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:08.802 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:04:08.567+0000 37792700 -1 osd.3 258 build_incremental_map_msg missing incremental map 258 2019-12-09T19:04:33.369 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:33.364+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:33.466 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:33.460+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:34.354 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:34.349+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:34.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:34.459+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:34.686 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:34.681+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:34.947 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:34.941+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:35.329 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:35.323+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:35.484 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:35.478+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:35.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:35.666+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:35.907 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:35.900+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:36.353 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:36.348+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:36.370 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:36.364+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:36.459 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:36.454+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:36.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:36.627+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:36.935 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:36.926+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:37.395 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:37.392+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:37.416 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:37.410+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:37.449 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:37.443+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:37.661 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:37.658+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:37.728 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:37.725+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:37.974 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:37.968+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:38.357 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:38.353+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:38.385 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:38.379+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:38.432 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:38.426+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:38.655 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:38.650+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:38.784 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:38.778+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:39.010 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:39.005+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:39.268 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:39.263+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:39.366 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:39.360+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:39.407 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:39.402+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:39.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:39.446+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:39.644 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:39.640+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:39.745 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:39.736+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:40.052 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:40.045+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:40.289 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:40.284+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:40.384 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:40.378+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:40.444 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:40.440+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:40.460 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:40.453+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:40.607 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:40.602+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:40.749 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:40.745+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:41.032 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:41.027+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:41.295 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:41.290+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:41.343 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:41.338+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:41.430 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:41.421+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:41.468 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:41.462+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:41.570 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:41.565+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:41.753 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:41.749+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:42.018 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:42.013+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:42.282 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:42.274+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:42.395 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:42.390+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:42.437 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:42.431+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:42.446 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:42.441+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:42.550 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:42.545+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:42.806 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:42.802+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:43.067 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:43.062+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:43.274 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:43.270+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:43.420 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:43.413+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:43.439 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:43.435+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:43.478 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:43.473+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:43.544 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:43.540+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:43.841 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:43.837+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:44.039 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:44.034+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:44.325 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:44.311+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:44.398 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:44.392+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:44.440 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:44.436+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:44.494 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:44.489+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:44.574 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:44.570+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:44.851 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:44.846+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:45.081 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:45.076+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:45.358 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:45.353+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:45.440 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:45.433+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:45.484 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:45.479+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:45.518 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:45.512+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:45.591 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:45.587+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:45.849 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:45.845+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:46.046 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:46.039+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:46.338 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:46.332+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:46.339 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:46.334+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:46.439 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:46.433+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:46.483 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:46.478+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:46.527 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:46.522+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:46.550 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:46.545+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:46.901 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:46.896+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:47.048 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:47.042+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:47.384 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:47.378+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:47.388 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:47.384+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:47.399 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:47.392+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:47.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:47.438+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:47.552 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:47.548+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:47.570 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:47.566+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:47.941 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:47.936+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:48.093 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:48.087+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:48.393 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:48.386+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:48.394 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:48.389+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:48.437 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:48.432+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:48.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:48.475+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:48.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:48.531+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:48.611 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:48.602+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:48.914 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:48.909+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:49.051 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:49.046+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:49.397 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:49.390+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:49.398 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:49.393+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:49.483 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:49.476+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:49.491 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:49.482+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:49.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:49.543+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:49.658 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:49.654+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:49.953 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:49.948+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:50.024 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:50.019+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:50.423 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:50.416+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:50.424 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:50.419+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:50.436 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:50.430+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:50.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:50.456+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:50.533 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:50.529+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:50.613 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:50.608+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:50.908 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:50.903+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:51.030 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:51.025+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:51.417 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:51.412+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:51.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:51.414+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:51.441 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:51.436+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:51.449 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:51.444+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:51.586 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:51.581+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:51.665 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:51.660+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:51.873 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:51.869+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:52.081 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:52.077+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:52.434 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:52.430+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:52.437 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:52.434+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:52.476 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:52.472+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:52.493 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:52.488+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:52.605 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:52.598+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:52.662 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:52.658+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:52.861 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:52.857+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:53.039 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:53.034+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:53.403 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:53.397+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:53.404 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:53.400+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:53.469 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:53.463+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:53.476 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:53.470+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:53.597 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:53.591+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:53.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:53.663+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:53.900 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:53.897+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:54.091 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:54.085+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:54.435 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:54.430+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:54.441 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:54.436+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:54.457 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:54.451+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:54.458 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:54.454+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:54.588 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:54.583+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:54.624 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:54.620+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:54.894 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:54.891+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:55.155 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:55.140+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:55.432 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:55.426+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:55.433 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:55.427+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:55.435 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:55.429+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:55.436 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:55.431+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:55.591 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:55.584+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:55.634 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:55.630+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:55.858 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:55.854+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:56.152 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:56.146+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:56.397 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:56.392+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:56.400 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:56.396+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:56.427 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:56.422+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:56.456 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:56.451+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:56.606 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:56.602+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:56.624 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:56.619+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:56.875 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:56.869+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:57.150 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:57.144+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:57.444 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:57.439+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:57.447 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:57.441+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:57.448 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:57.444+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:57.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:57.459+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:57.584 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:57.578+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:57.585 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:57.579+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:57.902 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:57.897+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:58.174 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:58.169+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:58.411 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:58.406+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:58.415 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:58.411+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:58.448 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:58.443+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:58.471 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:58.466+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:58.611 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:58.606+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:58.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:58.618+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:58.930 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:58.924+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:04:59.172 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:04:59.167+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:04:59.416 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:04:59.410+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:04:59.420 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:59.414+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:04:59.421 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:04:59.416+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:04:59.436 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:04:59.431+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:04:59.595 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:04:59.589+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:04:59.650 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:04:59.645+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:04:59.916 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:04:59.910+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:00.204 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:00.198+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:00.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:00.393+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:00.420 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:00.407+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:00.421 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:00.408+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:00.487 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:00.482+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:00.617 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:00.607+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:00.704 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:00.698+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:00.891 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:00.884+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:01.177 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:01.172+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:01.367 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:01.361+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:01.368 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:01.363+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:01.412 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:01.406+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:01.533 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:01.527+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:01.585 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:01.579+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:01.756 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:01.751+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:01.940 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:01.935+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:02.139 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:02.133+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:02.365 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:02.360+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:02.369 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:02.365+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:02.402 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:02.396+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:02.588 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:02.583+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:02.636 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:02.630+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:02.730 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:02.726+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:02.949 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:02.943+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:03.151 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:03.145+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:03.347 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:03.341+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:03.348 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:03.344+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:03.383 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:03.377+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:03.596 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:03.591+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:03.606 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:03.595+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:03.776 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:03.771+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:03.941 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:03.936+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:04.155 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:04.149+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:04.389 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:04.384+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:04.390 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:04.383+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:04.408 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:04.395+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:04.598 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:04.592+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:04.648 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:04.643+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:04.740 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:04.735+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:04.980 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:04.974+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:05.122 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:05:05.118+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e260 v260) 2019-12-09T19:05:05.133 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:05.128+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:05.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:05.391+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:05.415 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:05.409+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:05.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:05.414+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:05.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:05.615+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:05.667 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:05.662+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:05.711 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:05.706+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:06.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:05.995+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:06.120 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:06.114+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:06.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:06.359+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:06.387 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:06.382+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:06.393 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:06.387+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:06.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:06.614+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:06.673 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:06.667+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:06.739 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:06.733+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:07.050 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:07.044+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:07.136 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:07.131+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:07.322 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:07.317+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:07.377 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:07.371+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:07.377 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:07.373+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:07.598 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:07.593+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:07.701 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:07.696+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:07.778 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:07.773+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:08.022 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:08.016+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:08.140 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:08.135+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:08.365 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:08.359+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:08.370 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:08.364+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:08.371 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:08.367+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:08.604 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:08.599+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:08.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:08.722+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:08.806 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:08.801+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:09.074 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:09.068+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:09.183 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:09.177+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:09.375 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:09.367+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:09.391 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:09.387+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:09.395 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:09.391+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:09.639 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:09.632+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:09.760 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:09.755+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:09.765 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:09.760+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:10.066 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:10.061+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:10.146 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:05:10.139+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e260 v260) 2019-12-09T19:05:10.186 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:10.179+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:10.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:10.360+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:10.386 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:10.379+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:10.386 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:10.382+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:10.594 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:10.588+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:10.726 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:10.721+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:10.727 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:10.722+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:11.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:11.051+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:11.169 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:11.164+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:11.329 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:11.324+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:11.429 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:11.423+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:11.430 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:11.425+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:11.558 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:11.551+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:11.687 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:11.682+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:11.773 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:11.767+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:12.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:12.046+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:12.170 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:12.165+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:12.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:12.309+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:12.425 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:12.420+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:12.428 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:12.424+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:12.550 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:12.544+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:12.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:12.671+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:12.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:12.785+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:13.091 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:13.085+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:13.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:13.219+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:13.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:13.277+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:13.390 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:13.383+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:13.391 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:13.386+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:13.511 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:13.505+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:13.651 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:13.646+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:13.745 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:13.739+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:14.052 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:14.046+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:14.189 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:14.183+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:14.311 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:14.305+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:14.422 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:14.416+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:14.423 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:14.418+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:14.562 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:14.556+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:14.638 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:14.632+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:14.765 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:14.759+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:15.040 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:15.035+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:15.160 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:05:15.154+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e260 v260) 2019-12-09T19:05:15.241 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:15.236+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:15.296 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:15.291+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:15.443 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:15.437+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:15.444 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:15.439+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:15.596 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:15.591+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:15.639 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:15.633+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:15.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:15.762+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:16.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:16.056+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:16.226 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:16.221+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:16.301 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:16.295+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:16.476 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:16.471+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:16.480 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:16.476+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:16.610 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:16.605+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:16.634 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:16.630+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:16.738 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:16.732+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:17.106 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:17.100+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:17.276 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:17.270+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:17.333 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:17.328+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:17.497 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:17.492+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:17.500 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:17.496+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:17.628 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:17.622+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:17.644 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:17.639+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:17.715 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:17.705+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:18.126 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:18.120+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:18.272 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:18.266+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:18.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:18.303+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:18.549 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:18.544+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:18.553 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:18.549+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:18.637 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:18.631+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:18.665 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:18.659+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:18.667 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:18.662+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:19.173 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:19.166+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:19.232 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:19.226+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:19.351 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:19.346+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:19.554 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:19.546+0000 1694f700 -1 osd.4 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:04:05.693738+0000 (oldest deadline 2019-12-09T19:04:45.693738+0000) 2019-12-09T19:05:19.555 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:19.550+0000 1694f700 -1 osd.4 260 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:05:19.618 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:05:19.612+0000 1694f700 -1 osd.2 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:48.855604+0000 front 2019-12-09T19:03:48.985264+0000 (oldest deadline 2019-12-09T19:04:33.767456+0000) 2019-12-09T19:05:19.646 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:05:19.640+0000 1694f700 -1 osd.6 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:47.806127+0000 front 2019-12-09T19:03:47.772132+0000 (oldest deadline 2019-12-09T19:04:33.388811+0000) 2019-12-09T19:05:19.715 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:05:19.709+0000 1694f700 -1 osd.0 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:51.717413+0000 front 2019-12-09T19:03:51.626665+0000 (oldest deadline 2019-12-09T19:04:35.702079+0000) 2019-12-09T19:05:20.207 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:05:20.203+0000 1694f700 -1 osd.1 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:52.025896+0000 front 2019-12-09T19:03:51.845585+0000 (oldest deadline 2019-12-09T19:04:36.979693+0000) 2019-12-09T19:05:20.232 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:05:20.226+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e260 v260) 2019-12-09T19:05:20.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:05:20.232+0000 1694f700 -1 osd.7 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.575132+0000 front 2019-12-09T19:03:49.610134+0000 (oldest deadline 2019-12-09T19:04:34.860541+0000) 2019-12-09T19:05:20.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:05:20.337+0000 1694f700 -1 osd.5 260 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:03:49.252712+0000 front 2019-12-09T19:03:49.251387+0000 (oldest deadline 2019-12-09T19:04:33.031916+0000) 2019-12-09T19:05:20.685 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:05:20.594+0000 1694f700 -1 osd.4 261 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e258) 2019-12-09T19:06:50.532 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:42.382 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.541 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:42.391 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.550 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:42.375 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.554 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:42.379 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.557 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:42.382 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.560 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:42.385 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.836 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:42.661 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.898 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:42.747 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:50.917 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:42.742 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:06:52.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:52.236+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:52.386 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:52.381+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:53.158 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:53.153+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:53.251 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:53.246+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:53.425 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:53.420+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:53.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:53.572+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:53.724 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:53.720+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:54.083 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:54.077+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:54.210 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:54.205+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:54.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:54.266+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:54.410 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:54.406+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:54.575 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:54.561+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:54.740 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:54.735+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:55.122 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:55.116+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:55.221 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:55.216+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:55.319 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:55.313+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:55.418 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:55.412+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:55.544 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:55.539+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:55.768 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:55.763+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:56.119 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:56.113+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:56.196 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:56.192+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:56.276 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:56.271+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:56.413 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:56.408+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:56.569 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:56.563+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:56.792 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:56.787+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:57.173 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:57.169+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:57.246 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:57.242+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:57.283 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:57.280+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:57.417 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:57.413+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:57.616 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:57.610+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:57.794 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:57.790+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:58.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:58.180+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:58.251 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:58.247+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:58.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:58.287+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:58.393 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:58.390+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:58.591 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:58.586+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:58.800 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:58.794+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:06:59.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:06:59.193+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:06:59.261 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:06:59.256+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:06:59.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:06:59.268+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:06:59.421 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:06:59.418+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:06:59.575 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:06:59.570+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:06:59.839 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:06:59.835+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:00.210 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:00.204+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:00.223 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:00.218+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:00.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:00.236+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:00.392 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:00.388+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:00.591 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:00.586+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:00.875 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:00.870+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:01.194 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:01.189+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:01.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:01.232+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:01.252 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:01.248+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:01.420 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:01.416+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:01.587 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:01.582+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:01.893 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:01.889+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:02.228 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:02.223+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:02.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:02.267+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:02.305 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:02.299+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:02.413 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:02.409+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:02.562 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:02.557+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:02.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:02.864+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:03.186 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:03.180+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:03.293 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:03.284+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:03.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:03.310+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:03.395 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:03.391+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:03.568 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:03.563+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:03.879 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:03.874+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:04.168 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:04.163+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:04.323 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:04.315+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:04.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:04.335+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:04.437 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:04.432+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:04.535 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:04.529+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:04.877 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:04.864+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:05.142 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:05.136+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:05.290 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:05.283+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:05.345 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:05.340+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:05.408 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:05.404+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:05.565 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:05.560+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:05.861 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:05.857+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:06.069 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:57.919 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.072 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:57.922 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.103 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:06.099+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:06.113 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:57.937 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.121 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:45:57.971 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.122 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:57.947 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.165 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:57.989 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.174 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:57.998 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.206 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:45:58.029 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.213 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:45:58.037 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:07:06.302 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:06.299+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:06.333 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:06.328+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:06.445 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:06.440+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:06.530 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:06.526+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:06.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:06.864+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:07.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:07.121+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:07.256 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:07.253+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:07.357 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:07.347+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:07.418 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:07.414+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:07.558 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:07.553+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:07.870 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:07.861+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:08.084 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:08.079+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:08.247 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:08.242+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:08.347 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:08.343+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:08.426 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:08.422+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:08.547 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:08.542+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:08.858 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:08.854+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:09.068 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:09.060+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:09.265 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:09.254+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:09.381 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:09.376+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:09.424 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:09.419+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:09.518 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:09.513+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:09.852 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:09.848+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:10.096 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:10.091+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:10.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:10.268+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:10.349 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:10.338+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:10.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:10.417+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:10.479 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:10.474+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:10.829 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:10.825+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:11.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:11.104+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:11.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:11.223+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:11.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:11.295+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:11.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:11.410+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:11.484 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:11.478+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:11.817 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:11.813+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:12.102 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:12.097+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:12.282 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:12.277+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:12.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:12.315+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:12.451 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:12.447+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:12.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:12.489+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:12.785 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:12.780+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:13.086 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:13.080+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:13.335 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:13.326+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:13.345 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:13.339+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:13.414 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:13.407+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:13.482 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:13.478+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:13.822 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:13.812+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:14.120 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:14.114+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:14.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:14.296+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:14.302 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:14.298+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:14.403 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:14.400+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:14.504 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:14.499+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:14.800 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:14.796+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:15.144 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:15.139+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:15.261 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:15.255+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:15.358 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:15.352+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:15.379 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:15.374+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:15.468 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:15.463+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:15.776 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:15.768+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:16.106 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:16.100+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:16.269 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:16.265+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:16.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:16.336+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:16.390 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:16.386+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:16.451 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:16.445+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:16.762 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:16.758+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:17.108 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:17.103+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:17.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:17.221+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:17.324 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:17.319+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:17.388 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:17.384+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:17.466 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:17.461+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:17.813 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:17.808+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:18.165 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:18.160+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:18.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:18.232+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:18.377 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:18.372+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:18.441 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:18.437+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:18.449 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:18.445+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:18.827 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:18.822+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:19.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:19.137+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:19.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:19.260+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:19.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:19.395+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:19.402 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:19.398+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:19.411 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:19.407+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:19.809 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:19.805+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:20.100 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:20.095+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:20.248 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:20.242+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:20.376 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:20.372+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:20.376 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:20.372+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:20.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:20.418+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:20.792 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:20.787+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:21.150 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:21.145+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:21.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:21.261+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:21.377 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:21.372+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:21.396 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:21.391+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:21.438 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:21.433+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:21.819 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:21.814+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:22.145 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:22.141+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:22.253 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:22.246+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:22.367 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:22.363+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:22.413 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:22.407+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:22.418 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:22.413+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:22.784 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:22.778+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:23.102 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:23.096+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:23.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:23.295+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:23.360 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:23.355+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:23.379 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:23.374+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:23.469 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:23.464+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:23.742 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:23.737+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:24.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:24.053+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:24.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:24.306+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:24.367 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:24.361+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:24.410 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:24.403+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:24.482 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:24.477+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:24.746 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:24.741+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:25.061 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:25.056+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:25.274 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:25.269+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:25.372 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:25.363+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:25.372 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:25.366+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:25.473 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:25.468+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:25.798 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:25.793+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:25.950 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:07:25.946+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e265 v265) 2019-12-09T19:07:26.090 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:26.085+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:26.248 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:26.243+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:26.397 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:26.392+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:26.420 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:26.414+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:26.522 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:26.517+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:26.756 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:26.751+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:27.101 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:27.096+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:27.220 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:27.215+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:27.399 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:27.394+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:27.412 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:07:27.405+0000 1694f700 -1 osd.1 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:12.388441+0000 front 2019-12-09T19:06:10.959215+0000 (oldest deadline 2019-12-09T19:07:27.376325+0000) 2019-12-09T19:07:27.415 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:27.409+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:27.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:27.568+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:27.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:27.706+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:28.114 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:28.109+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:07:28.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:07:28.259+0000 1694f700 -1 osd.6 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.132505+0000 front 2019-12-09T19:06:08.133553+0000 (oldest deadline 2019-12-09T19:06:52.211757+0000) 2019-12-09T19:07:28.418 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:07:28.411+0000 1694f700 -1 osd.1 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:12.388441+0000 front 2019-12-09T19:06:10.959215+0000 (oldest deadline 2019-12-09T19:07:27.376325+0000) 2019-12-09T19:07:28.428 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:07:28.423+0000 1694f700 -1 osd.2 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.497957+0000 front 2019-12-09T19:06:09.041616+0000 (oldest deadline 2019-12-09T19:06:51.732146+0000) 2019-12-09T19:07:28.431 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:07:28.427+0000 1694f700 -1 osd.0 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:09.254302+0000 front 2019-12-09T19:06:09.251931+0000 (oldest deadline 2019-12-09T19:06:53.124309+0000) 2019-12-09T19:07:28.594 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:07:28.589+0000 1694f700 -1 osd.7 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.874590+0000 front 2019-12-09T19:06:12.143769+0000 (oldest deadline 2019-12-09T19:06:53.481679+0000) 2019-12-09T19:07:28.759 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:07:28.755+0000 1694f700 -1 osd.3 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:11.695858+0000 front 2019-12-09T19:06:12.389767+0000 (oldest deadline 2019-12-09T19:06:52.858126+0000) 2019-12-09T19:07:29.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:07:29.105+0000 1694f700 -1 osd.5 265 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:06:08.139322+0000 front 2019-12-09T19:06:07.734460+0000 (oldest deadline 2019-12-09T19:06:53.568251+0000) 2019-12-09T19:08:06.377 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:08:06.190+0000 36790700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-12-09T19:08:06.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:06.335+0000 36b90700 -1 osd.1 269 build_incremental_map_msg missing incremental map 269 2019-12-09T19:08:06.477 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:08:06.285+0000 36790700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-12-09T19:08:06.677 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:08:06.498+0000 37792700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-12-09T19:08:27.387 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:27.381+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:28.063 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:28.059+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:28.086 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:28.082+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:28.372 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:28.367+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:28.672 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:28.667+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:29.024 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:29.019+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:29.070 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:29.064+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:29.338 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:29.332+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:29.645 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:29.635+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:30.036 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:30.029+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:30.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:30.092+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:30.352 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:30.347+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:30.407 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:30.402+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:30.655 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:30.650+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:30.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:30.984+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:31.089 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:31.077+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:31.375 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:31.370+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:31.401 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:31.395+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:31.446 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:31.441+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:31.686 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:31.682+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:32.016 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:32.012+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:32.040 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:32.036+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:32.372 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:32.367+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:32.423 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:32.418+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:32.453 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:32.447+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:32.677 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:32.672+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:33.021 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:33.015+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:33.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:33.056+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:33.347 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:33.341+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:33.462 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:33.457+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:33.465 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:33.460+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:33.646 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:33.641+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:33.997 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:33.994+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:34.063 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:34.059+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:34.349 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:34.343+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:34.468 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:34.462+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:34.494 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:34.489+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:34.686 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:34.682+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:34.959 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:34.954+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:35.025 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:35.020+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:35.356 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:35.350+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:35.453 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:35.447+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:35.534 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:35.529+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:35.714 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:35.709+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:35.928 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:35.923+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:35.992 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:35.985+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:36.395 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:36.389+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:36.485 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:36.479+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:36.582 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:36.577+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:36.693 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:36.687+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:36.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:36.882+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:36.958 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:36.954+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:37.363 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:37.358+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:37.500 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:37.495+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:37.557 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:37.552+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:37.690 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:37.685+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:37.882 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:37.876+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:37.940 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:37.935+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:38.239 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:38.235+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:38.376 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:38.370+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:38.504 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:38.498+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:38.538 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:38.532+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:38.654 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:38.648+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:38.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:38.842+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:38.972 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:38.966+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:39.294 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:39.289+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:39.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:39.385+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:39.518 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:39.512+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:39.522 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:39.517+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:39.666 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:39.661+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:39.885 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:39.880+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:39.969 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:39.963+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:40.292 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:40.287+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:40.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:40.406+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:40.524 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:40.519+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:40.561 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:40.556+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:40.647 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:40.643+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:40.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:40.861+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:40.974 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:40.969+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:41.264 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:41.256+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:41.409 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:41.404+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:41.515 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:41.509+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:41.530 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:41.524+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:41.696 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:41.691+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:41.822 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:41.817+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:42.026 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:42.021+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:42.237 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:42.229+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:42.399 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:42.392+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:42.516 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:42.511+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:42.519 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:42.515+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:42.698 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:42.693+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:42.874 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:42.869+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:42.993 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:42.988+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:43.214 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:43.209+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:43.388 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:43.383+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:43.485 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:43.479+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:43.516 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:43.510+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:43.671 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:43.666+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:43.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:43.916+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:44.026 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:44.022+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:44.221 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:44.216+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:44.365 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:44.359+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:44.450 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:44.444+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:44.507 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:44.501+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:44.681 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:44.675+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:44.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:44.916+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:44.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:44.987+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:45.204 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:45.198+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:45.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:45.200+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:45.406 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:45.400+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:45.415 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:45.410+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:45.505 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:45.497+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:45.670 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:45.664+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:45.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:45.878+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:45.955 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:45.950+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:46.209 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:46.203+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:46.210 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:46.205+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:46.430 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:46.425+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:46.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:46.448+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:46.531 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:46.526+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:46.675 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:46.670+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:46.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:46.902+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:46.970 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:46.964+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:47.261 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:47.256+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:47.266 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:47.262+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:47.440 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:47.430+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:47.450 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:47.445+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:47.547 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:47.541+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:47.684 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:47.679+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:47.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:08:47.919+0000 1694f700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.994980+0000 front 2019-12-09T19:07:43.141487+0000 (oldest deadline 2019-12-09T19:08:27.023804+0000) 2019-12-09T19:08:47.992 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:47.988+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:48.230 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:48.224+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:48.231 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:48.226+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:48.416 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:48.410+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:48.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:48.469+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:48.599 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:48.593+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:48.706 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:48.700+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:48.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:48.992+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:49.194 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:49.189+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:49.196 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:49.193+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:49.421 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:49.416+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:49.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:49.449+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:49.630 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:49.626+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:49.741 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:49.735+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:50.007 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:50.001+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:50.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:50.200+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:50.208 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:50.204+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:50.382 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:50.375+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:50.490 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:50.483+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:50.597 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:50.593+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:50.759 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:50.754+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:51.042 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:51.037+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:51.259 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:51.252+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:51.260 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:51.255+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:51.388 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:51.383+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:51.463 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:51.457+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:51.640 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:51.633+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:51.754 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:51.747+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:52.092 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:52.087+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:52.297 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:52.291+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:52.297 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:52.293+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:52.422 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:52.415+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:52.463 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:52.458+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:52.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:52.599+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:52.771 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:52.766+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:53.099 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:53.095+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:53.309 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:53.303+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:53.310 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:53.305+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:53.388 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:53.383+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:53.441 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:53.436+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:53.657 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:53.651+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:53.784 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:53.778+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:54.061 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:54.056+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:54.276 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:54.270+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:54.277 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:54.272+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:54.353 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:54.347+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:54.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:54.442+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:54.676 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:54.671+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:54.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:54.784+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:55.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:55.098+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:55.281 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:55.271+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:55.281 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:55.272+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:55.352 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:55.347+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:55.505 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:55.494+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:55.663 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:55.657+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:55.782 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:55.777+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:56.130 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:56.125+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:56.301 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:56.295+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:56.302 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:56.298+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:56.391 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:56.386+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:56.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:56.469+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:56.643 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:56.635+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:56.824 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:56.817+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:57.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:57.109+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:57.267 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:57.259+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:57.275 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:57.261+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:57.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:57.373+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:57.524 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:57.519+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:57.631 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:57.626+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:57.871 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:57.866+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:58.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:58.086+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:58.221 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:58.215+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:58.221 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:58.217+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:58.431 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:58.426+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:58.525 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:58.519+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:58.669 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:58.662+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:58.831 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:58.826+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:08:59.129 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:08:59.124+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:08:59.200 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:59.195+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:08:59.203 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:08:59.199+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:08:59.425 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:08:59.419+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:08:59.521 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:08:59.513+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:08:59.661 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:08:59.655+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:08:59.845 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:08:59.842+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:00.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:00.091+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:00.167 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:00.162+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:00.171 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:00.167+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:00.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:00.392+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:00.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:00.536+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:00.640 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:00.634+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:00.871 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:00.866+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:01.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:01.084+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:01.143 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:01.136+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:01.144 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:01.139+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:01.248 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:09:01.243+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e270 v270) 2019-12-09T19:09:01.372 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:01.366+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:01.527 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:01.521+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:01.686 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:01.681+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:01.905 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:01.900+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:02.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:02.068+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:02.195 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:02.190+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:02.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:02.195+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:02.404 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:02.398+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:02.508 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:02.502+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:02.744 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:02.738+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:02.891 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:02.887+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:03.059 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:03.054+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:03.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:03.198+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:03.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:03.201+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:03.373 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:03.368+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:03.481 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:03.475+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:03.731 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:03.726+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:03.895 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:03.889+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:04.065 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:04.060+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:04.190 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:04.186+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:04.190 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:04.187+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:04.331 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:04.326+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:04.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:04.456+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:04.789 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:04.784+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:04.870 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:04.864+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:05.019 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:05.015+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:05.151 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:05.146+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:05.152 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:05.148+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:05.337 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:05.333+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:05.483 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:05.478+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:05.759 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:05.755+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:05.880 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:05.877+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:05.985 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:05.981+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:06.164 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:06.159+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:06.167 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:06.162+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:06.262 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:09:06.260+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e270 v270) 2019-12-09T19:09:06.325 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:06.320+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:06.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:06.434+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:06.754 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:06.749+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:06.872 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:06.868+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:07.023 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:07.019+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:07.159 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:07.152+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:07.159 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:07.155+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:07.339 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:07.334+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:07.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:07.389+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:07.755 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:07.750+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:07.892 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:07.886+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:08.062 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:08.059+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:08.206 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:08.200+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:08.209 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:08.205+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:08.351 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:08.346+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:08.371 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:08.365+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:08.733 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:08.727+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:08.940 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:08.935+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:09.050 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:09.046+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:09.269 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:09.255+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:09.270 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:09.258+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:09.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:09.304+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:09.424 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:09.419+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:09.695 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:09.686+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:09.929 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:09.922+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:10.009 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:10.004+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:10.248 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:10.242+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:10.252 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:10.247+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:10.274 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:10.269+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:10.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:10.385+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:10.740 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:10.734+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:10.941 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:10.936+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:11.015 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:11.010+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:11.236 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:11.227+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:11.238 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:11.233+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:11.279 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:09:11.276+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e270 v270) 2019-12-09T19:09:11.296 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:11.290+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:11.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:11.439+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:11.788 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:11.783+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:11.970 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:11.966+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:11.983 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:11.979+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:12.290 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:12.284+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:12.291 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:12.286+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:12.321 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:12.315+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:12.484 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:12.477+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:12.758 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:12.752+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:12.958 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:12.953+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:12.974 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:12.969+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:13.268 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:13.250+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:13.269 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:13.252+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:13.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:13.349+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:13.472 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:13.466+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:13.722 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:13.715+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:13.959 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:13.955+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:13.990 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:13.986+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:14.264 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:14.257+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:14.264 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:14.260+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:14.357 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:14.350+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:14.439 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:14.433+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:14.737 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:14.731+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:14.916 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:14.911+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:14.983 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:14.979+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:15.305 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:15.300+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:15.309 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:15.305+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:15.368 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:15.362+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:15.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:15.389+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:15.713 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:09:15.709+0000 1694f700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:44.550084+0000 front 2019-12-09T19:07:44.526388+0000 (oldest deadline 2019-12-09T19:08:30.025228+0000) 2019-12-09T19:09:15.941 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:09:15.936+0000 1694f700 -1 osd.1 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.642824+0000 front 2019-12-09T19:07:42.778130+0000 (oldest deadline 2019-12-09T19:08:28.531109+0000) 2019-12-09T19:09:15.944 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:09:15.940+0000 1694f700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:42.962802+0000 front 2019-12-09T19:07:42.883786+0000 (oldest deadline 2019-12-09T19:08:27.474337+0000) 2019-12-09T19:09:16.304 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:16.298+0000 1694f700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:08:04.234821+0000 (oldest deadline 2019-12-09T19:08:44.234821+0000) 2019-12-09T19:09:16.305 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:09:16.300+0000 1694f700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e269) 2019-12-09T19:09:16.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:09:16.350+0000 1694f700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:43.855603+0000 front 2019-12-09T19:07:43.906009+0000 (oldest deadline 2019-12-09T19:08:26.749458+0000) 2019-12-09T19:09:16.365 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:09:16.359+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e270 v270) 2019-12-09T19:09:16.423 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:09:16.419+0000 1694f700 -1 osd.6 270 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:07:45.569273+0000 front 2019-12-09T19:07:45.828329+0000 (oldest deadline 2019-12-09T19:08:30.673585+0000) 2019-12-09T19:09:40.697 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:09:40.687+0000 36f91700 -1 osd.3 270 build_incremental_map_msg missing incremental map 269 2019-12-09T19:10:41.799 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:33.649 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:41.802 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:33.652 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:41.933 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:33.758 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:41.940 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:33.765 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:42.053 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:33.877 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:42.061 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:33.886 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:42.082 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:33.932 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:42.152 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:33.977 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:42.430 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:34.254 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:47.280 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:47.275+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:48.276 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:48.271+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:48.327 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:48.321+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:48.329 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:48.325+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:48.981 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:40.831 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:48.989 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:40.839 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:48.999 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:40.824 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.002 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:40.825 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.004 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:40.829 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.005 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:40.830 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.022 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:49:40.872 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.030 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:49:40.854 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.032 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:49:40.857 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:10:49.208 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:49.203+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:49.286 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:49.282+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:49.305 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:49.302+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:49.306 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:49.302+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:50.174 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:50.168+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:50.176 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:50.172+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:50.282 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:50.278+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:50.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:50.283+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:50.319 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:50.315+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:51.162 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:51.157+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:51.163 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:51.158+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:51.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:51.283+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:51.302 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:51.297+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:51.348 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:51.344+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:51.514 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:51.509+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:51.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:51.904+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:52.140 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:52.134+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:52.150 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:52.146+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:52.317 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:52.312+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:52.332 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:52.327+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:52.356 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:52.352+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:52.554 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:52.549+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:52.892 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:52.887+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:53.104 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:53.100+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:53.169 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:53.164+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:53.349 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:53.341+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:53.349 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:53.344+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:53.356 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:53.351+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:53.519 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:53.513+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:53.860 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:53.856+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:54.080 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:54.076+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:54.142 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:54.137+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:54.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:54.296+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:54.351 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:54.341+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:54.352 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:54.345+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:54.526 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:54.522+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:54.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:54.842+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:55.043 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:55.038+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:55.152 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:55.146+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:55.267 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:55.262+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:55.327 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:55.322+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:55.379 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:55.373+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:55.548 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:55.541+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:55.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:55.897+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:56.041 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:56.030+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:56.150 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:56.146+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:56.247 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:56.244+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:56.342 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:56.336+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:56.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:56.389+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:56.526 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:56.520+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:56.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:56.935+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:56.995 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:56.990+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:57.194 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:57.189+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:57.235 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:57.231+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:57.305 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:57.299+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:57.363 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:57.358+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:57.518 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:57.513+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:57.959 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:57.955+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:57.966 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:57.962+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:58.219 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:58.215+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:58.238 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:58.233+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:58.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:58.277+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:58.328 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:58.323+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:58.511 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:58.506+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:58.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:58.918+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:58.954 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:58.948+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:10:59.238 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:10:59.232+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:10:59.250 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:10:59.243+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:10:59.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:10:59.257+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:10:59.297 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:10:59.293+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:10:59.547 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:10:59.543+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:10:59.927 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:10:59.916+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:10:59.998 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:10:59.993+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:00.208 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:00.202+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:00.269 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:00.262+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:00.279 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:00.273+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:00.290 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:00.284+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:00.574 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:00.569+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:00.971 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:00.966+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:00.993 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:00.989+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:01.179 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:01.174+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:01.261 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:01.257+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:01.292 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:01.287+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:01.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:01.321+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:01.611 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:01.605+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:01.951 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:01.944+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:02.001 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:01.996+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:02.227 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:02.221+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:02.263 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:02.257+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:02.335 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:02.327+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:02.336 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:02.328+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:02.598 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:02.593+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:02.960 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:02.950+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:03.004 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:02.995+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:03.218 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:03.213+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:03.229 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:03.224+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:03.373 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:03.368+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:03.383 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:03.378+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:03.613 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:03.608+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:03.974 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:03.969+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:04.053 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:04.049+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:04.234 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:04.229+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:04.244 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:04.240+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:04.394 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:04.389+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:04.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:04.394+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:04.637 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:04.632+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:04.992 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:04.987+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:05.061 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:05.056+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:05.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:05.208+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:05.219 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:05.215+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:05.409 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:05.404+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:05.453 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:05.448+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:05.657 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:05.652+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:05.972 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:05.965+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:06.046 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:06.042+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:06.191 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:06.187+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:06.208 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:06.202+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:06.439 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:06.434+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:06.466 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:06.461+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:06.685 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:06.679+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:06.999 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:06.994+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:07.014 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:07.009+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:07.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:07.217+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:07.240 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:07.234+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:07.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:07.466+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:07.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:07.470+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:07.685 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:07.680+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:08.009 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:08.005+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:08.064 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:08.053+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:08.195 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:08.190+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:08.201 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:08.195+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:08.468 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:08.456+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:08.483 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:08.478+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:08.708 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:08.703+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:09.025 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:09.019+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:09.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:09.033+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:09.203 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:09.198+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:09.246 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:09.242+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:09.424 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:09.420+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:09.523 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:09.519+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:09.749 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:09.743+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:09.991 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:09.985+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:10.055 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:10.049+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:10.240 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:10.235+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:10.280 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:10.274+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:10.446 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:10.443+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:10.504 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:10.500+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:10.764 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:10.757+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:10.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:10.958+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:11.096 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:11.091+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:11.258 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:11.255+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:11.291 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:11.284+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:11.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:11.478+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:11.482 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:11.478+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:11.728 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:11.723+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:11.996 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:11.991+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:12.080 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:12.076+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:12.213 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:12.210+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:12.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:12.334+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:12.461 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:12.457+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:12.506 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:12.501+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:12.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:12.705+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:12.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:12.974+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:13.074 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:13.069+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:13.237 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:13.232+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:13.327 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:13.317+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:13.480 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:13.473+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:13.481 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:13.475+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:13.742 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:13.737+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:14.007 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:14.001+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:14.127 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:14.122+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:14.282 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:14.277+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:14.334 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:14.329+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:14.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:14.456+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:14.515 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:14.510+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:14.766 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:14.760+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:14.971 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:14.965+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:15.138 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:15.132+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:15.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:15.279+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:15.293 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:15.288+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:15.425 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:15.420+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:15.562 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:15.557+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:15.732 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:15.726+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:16.023 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:16.017+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:16.100 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:16.095+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:16.286 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:16.277+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:16.302 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:16.297+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:16.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:16.449+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:16.612 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:16.606+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:16.741 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:16.735+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:17.030 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:17.025+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:17.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:17.123+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:17.294 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:17.287+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:17.301 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:17.296+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:17.494 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:17.489+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:17.581 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:17.575+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:17.754 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:17.748+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:18.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:18.017+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:18.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:18.126+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:18.292 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:18.286+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:18.306 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:18.301+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:18.466 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:18.461+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:18.542 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:18.537+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:18.747 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:18.741+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:19.065 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:19.059+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:19.107 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:19.101+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:19.274 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:19.267+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:19.304 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:19.299+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:19.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:19.442+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:19.515 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:19.510+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:19.743 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:19.737+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:20.029 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:20.024+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:20.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:20.135+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:20.284 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:20.278+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:20.346 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:20.341+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:20.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:20.450+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:20.553 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:20.547+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:20.772 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:20.766+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:21.046 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:21.041+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:21.158 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:21.152+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:21.265 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:21.259+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:21.370 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:21.366+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:21.483 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:21.478+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:21.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:21.601+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:21.783 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:21.778+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:21.926 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:11:21.920+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e276 v276) 2019-12-09T19:11:22.009 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:22.001+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:22.198 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:22.193+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:22.259 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:22.254+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:22.391 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:22.386+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:22.453 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:22.448+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:22.618 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:22.612+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:22.829 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:22.824+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:22.969 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:22.961+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:23.252 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:23.244+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:23.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:23.292+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:23.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:23.417+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:23.466 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:23.461+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:23.604 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:23.600+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:23.831 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:23.826+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:23.982 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:23.975+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:24.292 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:24.286+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:24.292 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:24.286+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:24.442 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:24.434+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:24.453 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:24.449+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:24.597 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:24.591+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:24.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:24.863+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:24.978 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:24.973+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:25.244 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:25.239+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:25.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:25.339+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:25.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:25.455+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:25.484 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:25.479+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:25.565 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:25.559+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:25.913 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:25.909+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:25.990 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:25.986+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:26.219 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:11:26.215+0000 1694f700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.878848+0000 front 2019-12-09T19:10:05.154994+0000 (oldest deadline 2019-12-09T19:10:49.673916+0000) 2019-12-09T19:11:26.336 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:26.333+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:11:26.454 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:11:26.449+0000 1694f700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.863101+0000 front 2019-12-09T19:10:05.415579+0000 (oldest deadline 2019-12-09T19:10:48.953449+0000) 2019-12-09T19:11:26.497 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:11:26.492+0000 1694f700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.450463+0000 front 2019-12-09T19:10:03.184624+0000 (oldest deadline 2019-12-09T19:10:47.005639+0000) 2019-12-09T19:11:26.547 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:11:26.543+0000 1694f700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.432244+0000 front 2019-12-09T19:10:03.532685+0000 (oldest deadline 2019-12-09T19:10:47.989842+0000) 2019-12-09T19:11:26.879 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:11:26.876+0000 1694f700 -1 osd.3 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:05.925443+0000 front 2019-12-09T19:10:05.915317+0000 (oldest deadline 2019-12-09T19:10:50.993349+0000) 2019-12-09T19:11:26.993 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:11:26.987+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e276 v276) 2019-12-09T19:11:27.021 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:11:27.016+0000 1694f700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:07.084112+0000 front 2019-12-09T19:10:07.041358+0000 (oldest deadline 2019-12-09T19:10:51.611949+0000) 2019-12-09T19:11:27.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:11:27.321+0000 1694f700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:10:03.213561+0000 front 2019-12-09T19:10:03.267201+0000 (oldest deadline 2019-12-09T19:10:47.272436+0000) 2019-12-09T19:12:25.256 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:25.251+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:25.903 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:25.897+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:26.273 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:26.268+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:26.885 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:26.879+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:27.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:27.288+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:27.574 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:27.567+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:27.904 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:27.896+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:28.063 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:28.058+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:28.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:28.255+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:28.531 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:28.526+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:28.859 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:28.853+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:29.097 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:29.091+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:29.296 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:29.290+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:29.533 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:29.528+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:29.855 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:29.849+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:30.090 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:30.085+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:30.108 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:30.102+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:30.169 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:30.165+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:30.333 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:30.328+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:30.392 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:30.386+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:30.577 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:30.572+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:30.887 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:30.882+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:31.076 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:31.070+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:31.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:31.081+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:31.129 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:31.125+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:31.351 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:31.344+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:31.422 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:31.416+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:31.617 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:31.611+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:31.940 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:31.935+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:32.031 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:32.026+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:32.076 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:32.069+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:32.153 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:32.148+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:32.325 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:32.320+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:32.396 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:32.391+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:32.596 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:32.589+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:32.983 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:32.977+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:32.985 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:32.980+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:33.069 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:33.063+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:33.197 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:33.192+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:33.313 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:33.308+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:33.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:33.399+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:33.559 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:33.554+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:33.947 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:33.940+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:33.972 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:33.967+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:34.055 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:34.048+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:34.241 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:34.237+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:34.369 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:34.362+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:34.388 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:34.381+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:34.540 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:34.535+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:34.941 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:34.936+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:34.972 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:34.968+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:35.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:35.072+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:35.246 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:35.240+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:35.247 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:35.243+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:35.353 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:35.348+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:35.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:35.394+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:35.520 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:35.514+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:35.952 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:35.946+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:35.981 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:35.979+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:36.127 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:36.122+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:36.254 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:36.250+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:36.256 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:36.254+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:36.391 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:36.386+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:36.439 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:36.433+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:36.537 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:36.532+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:36.950 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:36.944+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:36.997 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:36.992+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:37.174 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:37.168+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:37.300 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:37.296+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:37.303 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:37.300+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:37.413 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:37.408+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:37.478 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:37.472+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:37.505 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:37.499+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:37.909 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:37.902+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:38.011 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:38.004+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:38.175 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:38.168+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:38.323 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:38.316+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:38.323 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:38.319+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:38.369 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:38.364+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:38.515 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:38.509+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:38.520 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:38.514+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:38.928 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:38.922+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:39.009 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:39.003+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:39.226 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:39.221+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:39.332 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:39.327+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:39.336 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:39.331+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:39.400 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:39.393+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:39.469 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:39.464+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:39.481 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:39.476+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:39.955 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:39.950+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:40.059 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:40.053+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:40.267 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:40.261+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:40.358 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:40.347+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:40.385 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:40.380+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:40.388 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:40.384+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:40.468 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:40.464+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:40.486 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:40.480+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:40.946 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:40.941+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:41.032 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:41.026+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:41.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:41.307+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:41.314 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:41.309+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:41.419 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:41.414+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:41.423 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:41.418+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:41.450 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:41.444+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:41.462 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:41.457+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:41.953 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:41.947+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:41.996 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:41.990+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:42.288 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:42.285+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:42.308 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:42.303+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:42.445 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:42.442+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:42.467 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:42.461+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:42.470 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:42.466+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:42.488 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:42.483+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:42.959 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:42.946+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:42.964 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:42.959+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:43.264 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:43.258+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:43.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:43.259+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:43.440 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:43.434+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:43.440 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:43.436+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:43.500 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:43.494+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:43.502 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:43.496+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:43.921 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:43.916+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:43.946 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:43.941+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:44.281 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:44.276+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:44.293 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:44.287+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:44.410 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:44.403+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:44.411 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:44.406+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:44.469 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:44.464+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:44.491 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:44.486+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:44.938 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:44.933+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:44.971 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:44.966+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:45.262 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:45.257+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:45.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:45.305+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:45.426 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:45.419+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:45.426 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:45.422+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:45.451 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:45.447+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:45.462 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:45.458+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:45.975 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:45.969+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:46.003 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:45.997+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:46.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:46.259+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:46.298 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:46.291+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:46.411 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:46.405+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:46.412 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:46.407+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:46.459 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:46.455+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:46.486 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:46.482+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:46.975 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:46.967+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:47.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:47.003+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:47.276 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:47.271+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:47.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:47.285+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:47.427 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:47.415+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:47.430 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:47.419+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:47.451 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:47.447+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:47.484 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:47.481+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:48.005 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:47.999+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:48.028 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:48.023+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:48.248 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:48.244+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:48.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:48.309+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:48.387 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:48.382+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:48.390 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:48.386+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:48.482 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:48.477+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:48.489 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:48.485+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:49.027 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:49.021+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:49.035 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:49.030+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:49.246 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:49.241+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:49.353 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:49.343+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:49.436 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:49.429+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:49.437 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:49.432+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:49.505 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:49.500+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:49.538 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:49.533+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:50.035 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:50.030+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:50.049 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:50.045+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:50.268 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:50.264+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:50.311 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:50.304+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:50.399 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:50.394+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:50.403 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:50.399+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:50.476 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:50.473+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:50.590 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:50.583+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:51.034 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:51.029+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:51.091 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:51.085+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:51.222 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:51.218+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:51.332 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:51.324+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:51.412 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:51.405+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:51.413 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:51.408+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:51.508 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:51.502+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:51.636 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:51.632+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:52.049 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:52.044+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:52.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:52.128+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:52.269 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:52.264+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:52.343 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:52.336+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:52.456 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:52.449+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:52.457 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:52.452+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:52.533 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:52.529+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:52.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:52.672+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:53.081 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:53.076+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:53.109 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:53.104+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:53.253 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:53.249+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:53.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:53.307+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:53.474 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:53.467+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:53.475 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:53.469+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:53.557 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:53.552+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:53.643 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:53.638+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:54.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:54.055+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:54.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:54.132+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:54.236 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:54.230+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:54.324 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:54.318+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:54.460 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:54.454+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:54.461 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:54.457+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:54.555 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:54.550+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:54.634 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:54.629+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:55.068 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:55.062+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:55.122 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:55.116+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:55.267 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:55.261+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:55.293 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:55.287+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:55.445 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:55.439+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:55.446 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:55.441+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:55.552 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:55.547+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:55.675 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:55.670+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:56.037 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:56.032+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:56.127 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:56.122+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:56.287 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:56.283+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:56.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:56.313+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:56.453 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:56.447+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:56.454 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:56.449+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:56.538 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:56.533+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:56.711 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:56.708+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:57.022 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:12:57.016+0000 1694f700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:41.184134+0000 front 2019-12-09T19:11:41.200525+0000 (oldest deadline 2019-12-09T19:12:25.123981+0000) 2019-12-09T19:12:57.139 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:57.133+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:57.257 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:57.253+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:57.290 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:57.285+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:57.351 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:12:57.344+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e282 v282) 2019-12-09T19:12:57.504 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:57.499+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:57.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:57.500+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:57.582 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:57.577+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:57.763 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:57.758+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:58.103 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:58.098+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:58.267 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:58.262+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:58.275 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:58.270+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:58.485 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:58.479+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:58.486 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:58.481+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:58.612 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:58.606+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:58.739 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:58.734+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:12:59.067 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:12:59.062+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:12:59.255 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:12:59.251+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:12:59.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:12:59.278+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:12:59.491 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:59.485+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:12:59.492 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:12:59.487+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:12:59.651 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:12:59.646+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:12:59.775 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:12:59.771+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:00.072 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:00.066+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:00.270 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:00.264+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:00.295 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:00.289+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:00.466 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:00.460+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:00.467 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:00.462+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:00.674 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:00.670+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:00.783 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:00.780+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:01.042 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:01.036+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:01.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:01.306+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:01.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:01.322+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:01.448 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:01.443+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:01.452 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:01.448+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:01.673 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:01.667+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:01.742 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:01.738+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:02.073 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:02.068+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:02.272 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:02.267+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:02.347 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:02.342+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:02.368 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:13:02.359+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e282 v282) 2019-12-09T19:13:02.423 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:02.418+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:02.426 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:02.422+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:02.637 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:02.632+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:02.721 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:02.717+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:03.039 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:03.034+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:03.251 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:03.248+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:03.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:03.371+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:03.417 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:03.410+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:03.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:03.412+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:03.655 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:03.650+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:03.720 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:03.714+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:04.070 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:04.065+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:04.235 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:04.231+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:04.385 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:04.380+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:04.386 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:04.382+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:04.399 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:04.395+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:04.663 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:04.659+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:04.679 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:04.675+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:05.108 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:05.103+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:05.209 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:05.205+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:05.391 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:05.385+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:05.395 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:05.390+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:05.433 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:05.428+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:05.635 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:05.631+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:05.711 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:05.708+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:06.146 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:06.140+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:06.216 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:06.211+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:06.395 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:06.389+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:06.416 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:06.409+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:06.417 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:06.412+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:06.674 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:13:06.669+0000 1694f700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.902946+0000 front 2019-12-09T19:11:44.905045+0000 (oldest deadline 2019-12-09T19:12:30.127908+0000) 2019-12-09T19:13:06.732 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:13:06.727+0000 1694f700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.473667+0000 front 2019-12-09T19:11:42.256164+0000 (oldest deadline 2019-12-09T19:12:26.922453+0000) 2019-12-09T19:13:07.170 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:13:07.163+0000 1694f700 -1 osd.6 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:44.726735+0000 front 2019-12-09T19:11:44.860385+0000 (oldest deadline 2019-12-09T19:12:29.995101+0000) 2019-12-09T19:13:07.173 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:13:07.169+0000 1694f700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:43.524825+0000 front 2019-12-09T19:11:43.360153+0000 (oldest deadline 2019-12-09T19:12:27.222405+0000) 2019-12-09T19:13:07.401 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:07.396+0000 1694f700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:11:54.854884+0000 (oldest deadline 2019-12-09T19:12:34.854884+0000) 2019-12-09T19:13:07.406 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:07.401+0000 1694f700 -1 osd.4 282 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:114778 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-12-09T19:13:07.444 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:13:07.439+0000 1694f700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:11:42.540514+0000 front 2019-12-09T19:11:42.786368+0000 (oldest deadline 2019-12-09T19:12:25.174619+0000) 2019-12-09T19:13:07.455 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:13:07.450+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e282 v282) 2019-12-09T19:13:45.070 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:45.032+0000 32788700 -1 osd.4 286 build_incremental_map_msg missing incremental map 286 2019-12-09T19:13:45.247 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:45.094+0000 3578e700 -1 osd.4 286 build_incremental_map_msg missing incremental map 286 2019-12-09T19:13:45.356 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:45.165+0000 31f87700 -1 osd.4 286 build_incremental_map_msg missing incremental map 286 2019-12-09T19:13:45.405 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:13:45.251+0000 37f93700 -1 osd.4 286 build_incremental_map_msg missing incremental map 286 2019-12-09T19:14:47.103 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:47.097+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:47.293 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:47.287+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:47.755 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:47.750+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:47.838 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:47.834+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:47.940 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:53:39.790 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.946 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:53:39.796 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.956 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:53:39.779 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.959 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:53:39.784 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.963 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:53:39.788 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:47.968 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:53:39.792 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:48.121 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:48.116+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:48.162 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:53:40.012 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:48.185 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:53:40.010 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:48.188 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:53:40.013 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:14:48.247 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:48.242+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:48.468 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:48.462+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:48.782 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:48.777+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:48.856 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:48.851+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:49.166 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:49.161+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:49.200 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:49.195+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:49.362 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:49.357+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:49.456 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:49.451+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:49.814 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:49.809+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:49.819 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:49.815+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:50.166 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:50.161+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:50.235 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:50.231+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:50.316 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:50.311+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:50.459 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:50.451+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:50.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:50.805+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:50.843 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:50.837+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:51.208 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:51.203+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:51.218 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:51.213+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:51.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:51.332+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:51.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:51.417+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:51.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:51.833+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:51.838 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:51.833+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:52.208 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:52.203+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:52.229 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:52.223+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:52.317 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:52.312+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:52.408 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:52.403+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:52.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:52.791+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:52.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:52.862+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:53.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:53.237+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:53.260 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:53.251+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:53.360 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:53.353+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:53.447 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:53.442+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:53.755 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:53.750+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:53.902 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:53.897+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:54.274 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:54.269+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:54.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:54.281+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:54.375 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:54.369+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:54.433 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:54.427+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:54.758 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:54.754+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:54.882 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:54.877+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:55.319 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:55.313+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:55.335 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:55.330+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:55.351 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:55.345+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:55.467 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:55.462+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:55.746 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:55.741+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:55.898 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:55.894+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:56.307 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:56.303+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:56.310 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:56.305+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:56.355 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:56.350+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:56.495 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:56.490+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:56.789 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:56.785+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:56.863 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:56.857+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:57.345 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:57.339+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:57.355 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:57.349+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:57.385 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:57.381+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:57.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:57.509+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:57.771 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:57.767+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:57.873 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:57.870+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:58.318 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:58.312+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:58.380 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:58.374+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:58.436 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:58.431+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:58.479 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:58.474+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:58.797 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:58.793+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:58.886 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:58.881+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:14:59.297 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:14:59.292+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:14:59.335 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:14:59.329+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:14:59.440 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:14:59.434+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:14:59.500 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:14:59.495+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:14:59.764 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:14:59.760+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:14:59.839 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:14:59.834+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:00.293 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:00.287+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:00.342 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:00.337+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:00.489 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:00.482+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:00.490 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:00.485+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:00.777 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:00.771+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:00.839 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:00.834+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:01.321 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:01.317+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:01.358 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:01.352+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:01.474 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:01.469+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:01.476 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:01.468+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:01.784 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:01.779+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:01.817 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:01.812+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:02.296 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:02.291+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:02.375 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:02.370+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:02.453 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:02.447+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:02.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:02.488+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:02.805 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:02.794+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:02.865 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:02.860+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:03.313 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:03.308+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:03.340 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:03.334+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:03.451 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:03.445+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:03.501 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:03.495+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:03.767 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:03.758+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:03.852 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:03.847+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:04.275 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:04.270+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:04.351 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:04.345+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:04.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:04.416+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:04.554 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:04.548+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:04.763 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:04.758+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:04.893 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:04.888+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:05.317 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:05.311+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:05.332 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:05.325+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:05.386 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:05.379+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:05.515 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:05.509+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:05.774 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:05.769+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:05.904 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:05.899+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:06.283 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:06.274+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:06.357 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:06.352+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:06.360 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:06.355+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:06.541 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:06.532+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:06.757 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:06.752+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:06.869 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:06.865+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:07.241 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:07.234+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:07.321 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:07.316+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:07.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:07.347+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:07.534 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:07.529+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:07.748 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:07.743+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:07.833 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:07.827+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:08.199 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:08.194+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:08.351 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:08.346+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:08.365 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:08.359+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:08.544 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:08.539+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:08.713 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:08.708+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:08.856 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:08.851+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:09.208 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:09.202+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:09.319 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:09.315+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:09.341 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:09.335+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:09.522 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:09.517+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:09.691 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:09.681+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:09.841 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:09.835+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:10.243 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:10.238+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:10.370 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:10.365+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:10.391 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:10.386+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:10.506 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:10.501+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:10.655 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:10.650+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:10.816 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:10.809+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:11.277 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:11.274+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:11.360 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:11.356+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:11.390 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:11.386+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:11.519 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:11.513+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:11.631 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:11.626+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:11.778 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:11.773+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:12.312 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:12.304+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:12.326 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:12.322+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:12.348 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:12.345+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:12.515 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:12.511+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:12.667 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:12.662+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:12.830 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:12.825+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:13.294 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:13.289+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:13.303 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:13.299+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:13.328 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:13.320+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:13.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:13.466+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:13.658 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:13.654+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:13.844 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:13.839+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:14.299 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:14.295+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:14.333 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:14.329+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:14.354 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:14.350+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:14.467 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:14.462+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:14.615 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:14.609+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:14.805 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:14.801+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:15.289 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:15.285+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:15.330 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:15.326+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:15.347 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:15.343+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:15.472 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:15.467+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:15.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:15.591+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:15.762 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:15.757+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:16.257 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:16.251+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:16.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:16.310+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:16.345 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:16.341+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:16.444 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:16.438+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:16.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:16.602+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:16.812 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:16.805+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:17.256 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:17.245+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:17.269 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:17.265+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:17.316 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:17.311+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:17.450 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:17.445+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:17.600 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:17.595+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:17.791 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:17.785+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:18.252 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:18.246+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:18.279 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:18.275+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:18.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:18.294+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:18.439 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:18.434+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:18.600 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:18.595+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:18.772 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:18.767+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:19.228 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:19.223+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:19.253 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:19.249+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:19.330 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:19.325+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:19.447 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:19.441+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:19.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:19.591+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:19.736 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:19.731+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:20.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:20.223+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:20.263 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:20.257+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:20.349 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:15:20.341+0000 1694f700 -1 osd.3 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.707148+0000 front 2019-12-09T19:14:03.778010+0000 (oldest deadline 2019-12-09T19:14:46.789833+0000) 2019-12-09T19:15:20.481 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:15:20.476+0000 1694f700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.172898+0000 front 2019-12-09T19:14:02.184867+0000 (oldest deadline 2019-12-09T19:14:47.699786+0000) 2019-12-09T19:15:20.638 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:15:20.633+0000 1694f700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.540901+0000 front 2019-12-09T19:14:02.789551+0000 (oldest deadline 2019-12-09T19:14:47.111568+0000) 2019-12-09T19:15:20.714 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:15:20.708+0000 1694f700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:03.167283+0000 front 2019-12-09T19:14:02.693418+0000 (oldest deadline 2019-12-09T19:14:47.351023+0000) 2019-12-09T19:15:20.723 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:15:20.716+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.202073+0000 front 2019-12-09T19:14:02.203203+0000 (oldest deadline 2019-12-09T19:15:20.052592+0000) 2019-12-09T19:15:21.202 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:15:21.198+0000 1694f700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:05.129117+0000 front 2019-12-09T19:14:05.109156+0000 (oldest deadline 2019-12-09T19:14:48.337428+0000) 2019-12-09T19:15:21.255 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:15:21.251+0000 1694f700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:04.044183+0000 front 2019-12-09T19:14:03.939232+0000 (oldest deadline 2019-12-09T19:14:46.826429+0000) 2019-12-09T19:15:21.755 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:15:21.749+0000 1694f700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:14:02.202073+0000 front 2019-12-09T19:14:02.203203+0000 (oldest deadline 2019-12-09T19:15:20.052592+0000) 2019-12-09T19:15:58.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:15:58.507+0000 32b88700 -1 osd.1 291 build_incremental_map_msg missing incremental map 291 2019-12-09T19:15:58.655 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:15:58.592+0000 34b8c700 -1 osd.1 291 build_incremental_map_msg missing incremental map 291 2019-12-09T19:15:58.675 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:15:58.628+0000 34b8c700 -1 osd.1 291 build_incremental_map_msg missing incremental map 291 2019-12-09T19:16:19.183 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:19.176+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:19.530 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:19.525+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:20.217 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:20.212+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:20.434 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:20.428+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:20.514 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:20.509+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:21.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:21.233+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:21.459 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:21.454+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:21.548 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:21.542+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:21.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:21.601+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:22.290 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:22.284+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:22.442 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:22.437+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:22.518 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:22.513+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:22.562 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:22.556+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:22.609 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:22.604+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:23.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:23.135+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:23.315 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:23.310+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:23.455 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:23.449+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:23.475 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:23.469+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:23.587 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:23.581+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:23.606 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:23.600+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:24.168 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:24.163+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:24.323 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:24.319+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:24.467 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:24.462+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:24.483 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:24.478+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:24.575 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:24.568+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:24.607 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:24.602+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:25.184 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:25.179+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:25.362 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:25.355+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:25.454 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:25.451+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:25.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:25.457+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:25.564 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:25.560+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:25.602 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:25.596+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:26.154 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:26.150+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:26.339 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:26.333+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:26.431 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:26.426+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:26.439 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:26.436+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:26.575 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:26.570+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:26.615 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:26.612+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:27.206 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:27.201+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:27.375 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:27.369+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:27.388 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:27.382+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:27.478 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:27.474+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:27.621 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:27.616+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:27.666 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:27.662+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:28.186 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:28.181+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:28.365 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:28.358+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:28.406 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:28.400+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:28.457 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:28.453+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:28.583 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:28.579+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:28.687 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:28.683+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:29.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:29.165+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:29.322 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:29.317+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:29.417 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:29.413+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:29.446 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:29.440+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:29.544 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:29.538+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:29.657 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:29.652+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:29.840 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:29.836+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:30.166 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:30.162+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:30.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:30.297+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:30.398 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:30.393+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:30.483 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:30.476+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:30.580 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:30.576+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:30.623 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:30.615+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:30.818 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:30.813+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:31.164 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:31.156+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:31.261 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:31.255+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:31.408 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:31.403+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:31.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:31.455+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:31.591 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:31.587+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:31.600 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:31.595+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:31.781 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:31.776+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:32.208 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:32.204+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:32.244 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:32.240+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:32.459 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:32.454+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:32.512 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:32.506+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:32.556 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:32.551+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:32.579 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:32.574+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:32.804 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:32.799+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:33.221 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:33.216+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:33.225 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:33.221+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:33.425 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:33.420+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:33.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:33.545+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:33.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:33.568+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:33.622 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:33.611+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:33.770 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:33.766+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:34.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:34.231+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:34.278 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:34.272+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:34.382 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:34.378+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:34.529 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:34.526+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:34.557 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:34.552+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:34.571 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:34.566+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:34.730 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:34.725+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:35.247 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:35.240+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:35.281 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:35.270+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:35.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:35.337+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:35.524 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:35.519+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:35.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:35.545+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:35.552 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:35.547+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:35.716 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:35.710+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:35.719 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:35.715+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:36.220 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:36.215+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:36.240 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:36.234+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:36.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:36.297+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:36.528 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:36.522+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:36.540 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:36.535+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:36.601 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:36.594+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:36.722 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:36.716+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:36.723 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:36.718+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:37.217 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:37.212+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:37.222 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:37.217+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:37.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:37.336+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:37.564 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:37.559+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:37.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:37.574+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:37.636 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:37.631+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:37.723 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:37.717+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:37.724 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:37.719+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:38.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:38.231+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:38.249 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:38.242+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:38.368 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:38.365+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:38.584 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:38.579+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:38.613 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:38.607+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:38.615 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:38.609+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:38.706 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:38.699+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:38.707 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:38.702+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:39.231 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:39.226+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:39.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:39.289+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:39.333 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:39.328+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:39.552 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:39.547+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:39.573 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:39.568+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:39.590 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:39.585+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:39.699 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:39.692+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:39.701 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:39.695+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:40.210 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:40.205+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:40.339 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:40.334+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:40.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:40.342+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:40.584 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:40.578+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:40.609 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:40.605+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:40.625 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:40.620+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:40.742 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:40.736+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:40.743 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:40.738+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:41.232 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:41.226+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:41.354 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:41.340+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:41.357 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:41.353+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:41.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:41.568+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:41.596 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:41.592+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:41.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:41.602+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:41.779 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:41.774+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:41.780 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:41.776+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:42.199 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:42.187+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:42.317 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:42.308+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:42.374 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:42.369+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:42.553 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:42.548+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:42.581 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:42.575+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:42.601 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:42.596+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:42.744 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:42.739+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:42.747 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:42.743+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:43.243 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:43.237+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:43.359 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:43.348+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:43.375 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:43.369+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:43.526 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:43.521+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:43.568 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:43.562+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:43.655 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:43.649+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:43.709 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:43.703+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:43.710 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:43.705+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:44.225 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:44.220+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:44.389 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:44.384+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:44.426 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:44.421+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:44.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:44.531+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:44.613 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:44.608+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:44.673 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:44.668+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:44.739 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:44.732+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:44.740 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:44.735+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:45.234 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:45.228+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:45.388 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:45.385+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:45.397 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:45.393+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:45.550 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:45.546+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:45.641 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:45.635+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:45.661 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:45.656+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:45.739 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:45.733+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:45.740 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:45.735+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:46.202 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:46.196+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:46.399 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:46.395+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:46.437 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:46.434+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:46.561 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:46.556+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:46.632 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:46.626+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:46.643 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:46.637+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:46.774 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:46.769+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:46.778 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:46.774+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:47.158 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:47.153+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:47.407 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:47.403+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:47.442 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:47.436+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:47.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:47.576+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:47.665 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:47.660+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:47.688 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:47.682+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:47.818 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:47.813+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:47.819 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:47.815+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:48.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:48.156+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:48.366 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:48.362+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:48.485 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:48.481+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:48.602 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:48.598+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:48.664 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:48.660+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:48.705 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:48.700+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:48.825 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:48.820+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:48.826 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:48.822+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:49.212 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:49.207+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:49.369 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:49.366+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:49.522 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:49.518+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:49.582 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:49.578+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:49.657 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:49.651+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:49.719 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:49.711+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:49.850 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:49.844+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:49.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:49.845+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:50.191 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:50.185+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:50.372 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:50.362+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:50.548 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:50.544+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:50.605 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:50.600+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:50.649 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:50.643+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:50.729 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:50.724+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:50.826 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:50.820+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:50.827 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:50.822+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:51.211 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:51.206+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:51.409 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:51.405+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:51.519 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:51.514+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:51.569 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:51.565+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:51.620 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:51.615+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:51.741 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:51.735+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:51.871 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:51.864+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:51.872 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:51.867+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:52.260 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:52.255+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:52.442 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:52.437+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:52.528 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:52.523+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:52.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:52.568+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:52.636 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:52.630+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:52.738 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:52.732+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:52.865 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:52.860+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:52.869 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:52.864+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:53.257 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:53.252+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:53.273 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:16:53.265+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e292 v292) 2019-12-09T19:16:53.488 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:53.483+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:53.520 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:53.514+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:53.566 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:53.562+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:53.656 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:53.650+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:53.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:53.724+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:53.908 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:53.902+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:53.909 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:53.904+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:54.295 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:54.289+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:54.454 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:54.449+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:54.496 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:54.484+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:54.560 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:54.555+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:54.621 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:54.615+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:54.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:54.724+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:54.886 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:54.879+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:54.887 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:54.882+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:55.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:55.272+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:55.498 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:55.493+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:55.517 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:55.513+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:55.536 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:55.532+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:55.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:55.600+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:55.769 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:55.762+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:55.890 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:55.885+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:55.893 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:55.889+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:56.289 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:56.283+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:56.506 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:56.501+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:56.508 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:56.503+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:56.518 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:56.513+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:56.637 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:56.631+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:56.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:56.805+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:56.911 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:56.900+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:56.911 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:56.905+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:57.255 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:57.250+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:57.474 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:57.470+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:57.486 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:57.482+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:57.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:57.516+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:57.627 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:57.622+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:57.851 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:57.846+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:57.900 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:57.894+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:57.901 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:57.896+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:58.281 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:58.276+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:58.298 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:16:58.285+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e292 v292) 2019-12-09T19:16:58.492 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:58.488+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:58.524 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:58.521+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:58.529 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:58.526+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:58.649 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:58.644+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:58.862 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:58.856+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:58.883 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:58.877+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:58.886 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:58.882+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:16:59.260 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:16:59.253+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:16:59.461 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:16:59.456+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:16:59.497 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:16:59.493+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:16:59.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:16:59.517+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:16:59.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:16:59.642+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:16:59.828 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:16:59.823+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:16:59.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:59.882+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:16:59.890 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:16:59.884+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:00.259 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:00.254+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:00.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:00.418+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:00.525 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:00.522+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:00.553 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:00.550+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:00.643 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:00.628+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:00.834 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:00.829+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:00.904 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:00.899+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:00.905 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:00.901+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:01.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:01.236+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:01.463 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:01.454+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:01.489 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:01.485+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:01.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:01.572+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:01.684 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:01.679+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:01.853 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:01.848+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:01.915 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:01.908+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:01.916 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:01.910+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:02.304 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:02.291+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:02.477 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:02.473+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:02.500 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:02.496+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:02.616 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:02.611+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:02.719 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:02.715+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:02.857 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:02.852+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:02.916 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:02.909+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:02.917 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:02.911+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:03.274 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:03.269+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:03.311 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:17:03.306+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e292 v292) 2019-12-09T19:17:03.444 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:03.440+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:03.507 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:03.503+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:03.652 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:03.648+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:03.760 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:03.755+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:03.905 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:03.894+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:03.911 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:03.906+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:03.912 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:03.908+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:04.328 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:04.322+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:04.488 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:04.483+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:04.524 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:04.520+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:04.620 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:04.615+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:04.763 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:04.758+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:04.910 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:04.903+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:04.962 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:04.955+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:04.967 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:04.960+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:05.351 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:05.345+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:05.482 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:05.478+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:05.511 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:05.506+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:05.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:05.643+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:05.718 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:05.713+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:05.952 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:05.946+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:05.996 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:05.990+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:05.997 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:05.992+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:06.359 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:06.353+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:06.521 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:06.516+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:06.526 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:06.522+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:06.697 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:06.691+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:06.699 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:06.694+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:06.987 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:06.981+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:07.035 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:07.030+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:07.040 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:07.035+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:07.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:07.315+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:07.511 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:07.506+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:07.565 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:07.560+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:07.662 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:07.656+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:07.727 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:07.722+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:07.985 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:07.980+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:08.048 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:08.042+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:08.049 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:08.044+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:08.331 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:08.325+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:08.352 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:17:08.346+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e292 v292) 2019-12-09T19:17:08.508 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:08.504+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:08.560 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:08.556+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:08.651 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:08.645+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:08.736 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:08.732+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:09.011 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:09.005+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:09.080 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:09.074+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:09.081 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:09.076+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:09.318 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:09.313+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:09.477 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:09.473+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:09.611 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:09.607+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:09.641 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:09.636+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:09.756 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:09.752+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:09.980 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:09.974+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:10.127 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:10.121+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:10.128 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:10.123+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:10.299 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:10.294+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:10.517 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:10.512+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:10.634 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:10.629+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:10.652 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:10.645+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:10.725 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:10.721+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:10.992 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:10.986+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:11.131 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:11.124+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:11.132 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:11.127+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:11.282 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:11.276+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:11.494 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:11.490+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:11.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:11.612+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:11.698 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:11.692+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:11.721 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:11.716+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:12.000 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:11.995+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:12.156 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:12.150+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:12.156 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:12.152+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:12.297 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:12.291+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:12.498 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:12.492+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:12.668 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:17:12.661+0000 1694f700 -1 osd.1 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:37.050943+0000 front 2019-12-09T19:15:37.784095+0000 (oldest deadline 2019-12-09T19:16:22.440788+0000) 2019-12-09T19:17:12.668 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:17:12.663+0000 1694f700 -1 osd.7 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:33.801331+0000 front 2019-12-09T19:15:33.990829+0000 (oldest deadline 2019-12-09T19:16:18.841934+0000) 2019-12-09T19:17:12.758 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:12.752+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:12.959 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:17:12.953+0000 1694f700 -1 osd.5 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.109437+0000 front 2019-12-09T19:15:38.108355+0000 (oldest deadline 2019-12-09T19:16:20.967791+0000) 2019-12-09T19:17:13.188 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:13.180+0000 1694f700 -1 osd.4 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:15:55.434294+0000 (oldest deadline 2019-12-09T19:16:35.434294+0000) 2019-12-09T19:17:13.188 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:17:13.184+0000 1694f700 -1 osd.4 292 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117017 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-12-09T19:17:13.324 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:17:13.319+0000 1694f700 -1 osd.6 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:35.479935+0000 front 2019-12-09T19:15:35.441059+0000 (oldest deadline 2019-12-09T19:16:19.476438+0000) 2019-12-09T19:17:13.434 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:17:13.429+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e292 v292) 2019-12-09T19:17:13.457 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:17:13.453+0000 1694f700 -1 osd.2 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:38.412496+0000 front 2019-12-09T19:15:38.645485+0000 (oldest deadline 2019-12-09T19:16:21.838050+0000) 2019-12-09T19:17:13.727 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:17:13.720+0000 1694f700 -1 osd.0 292 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:15:30.430578+0000 front 2019-12-09T19:15:30.431219+0000 (oldest deadline 2019-12-09T19:16:19.215223+0000) 2019-12-09T19:17:38.754 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:17:38.737+0000 34f8d700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-12-09T19:17:38.833 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:17:38.827+0000 32788700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-12-09T19:17:39.865 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:17:39.853+0000 37f93700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-12-09T19:18:39.459 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:39.453+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:39.753 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:39.748+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:40.154 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:40.149+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:40.479 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:40.472+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:40.763 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:40.756+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:41.162 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:41.156+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:41.467 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:41.461+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:41.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:41.467+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:41.736 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:41.731+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:41.782 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:41.778+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:42.197 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:42.193+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:42.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:42.441+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:42.501 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:42.492+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:42.744 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:42.738+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:42.758 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:42.752+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:43.229 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:43.223+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:43.413 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:43.408+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:43.515 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:43.508+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:43.775 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:43.770+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:43.791 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:43.787+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:44.272 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:44.268+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:44.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:44.431+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:44.562 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:44.557+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:44.767 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:44.762+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:44.782 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:44.777+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:45.172 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:45.167+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:45.273 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:45.255+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:45.412 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:45.407+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:45.527 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:45.522+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:45.733 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:45.727+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:45.737 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:45.731+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:46.164 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:46.158+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:46.312 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:46.307+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:46.418 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:46.412+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:46.532 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:46.526+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:46.738 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:46.733+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:46.771 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:46.766+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:47.200 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:47.193+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:47.301 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:47.295+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:47.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:47.441+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:47.570 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:47.565+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:47.728 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:47.717+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:47.729 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:47.724+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:47.788 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:47.782+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:48.226 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:48.219+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:48.289 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:48.284+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:48.429 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:48.423+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:48.488 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:40.337 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.501 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:40.350 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.519 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:40.343 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.520 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:40.344 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.523 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:40.348 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.528 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:40.352 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:48.590+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:48.723 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:48.714+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:48.732 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:40.582 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.978 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:40.591 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.979 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:40.598 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:18:48.979 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:48.759+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:48.979 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:48.762+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:49.259 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:49.252+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:49.327 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:49.322+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:49.416 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:49.411+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:49.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:49.614+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:49.672 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:49.667+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:49.741 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:49.736+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:49.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:49.782+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:50.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:50.242+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:50.288 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:50.284+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:50.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:50.465+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:50.671 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:50.664+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:50.674 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:50.666+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:50.745 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:50.741+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:50.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:50.825+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:51.285 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:51.280+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:51.332 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:51.326+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:51.460 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:51.450+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:51.665 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:51.659+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:51.687 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:51.681+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:51.771 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:51.766+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:51.827 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:51.822+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:52.274 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:52.268+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:52.288 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:52.282+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:52.435 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:52.429+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:52.668 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:52.662+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:52.712 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:52.707+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:52.781 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:52.777+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:52.879 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:52.874+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:53.266 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:53.261+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:53.306 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:53.301+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:53.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:53.420+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:53.630 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:53.624+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:53.739 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:53.734+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:53.758 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:53.753+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:53.897 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:53.891+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:54.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:54.287+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:54.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:54.310+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:54.478 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:54.474+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:54.622 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:54.618+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:54.713 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:54.709+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:54.740 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:54.735+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:54.894 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:54.890+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:55.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:55.286+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:55.346 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:55.342+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:55.513 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:55.508+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:55.658 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:55.653+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:55.702 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:55.697+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:55.788 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:55.779+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:55.865 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:55.859+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:56.276 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:56.272+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:56.388 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:56.383+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:56.535 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:56.530+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:56.690 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:56.685+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:56.737 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:56.727+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:56.757 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:56.752+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:56.852 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:56.847+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:57.310 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:57.306+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:57.410 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:57.404+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:57.542 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:57.537+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:57.731 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:57.726+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:57.746 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:57.741+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:57.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:57.806+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:57.815 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:57.810+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:58.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:58.281+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:58.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:58.452+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:58.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:58.550+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:58.749 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:58.744+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:58.761 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:58.745+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:58.782 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:58.777+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:58.865 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:58.859+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:18:59.294 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:18:59.290+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:18:59.497 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:18:59.490+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:18:59.568 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:18:59.565+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:18:59.710 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:18:59.705+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:18:59.741 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:18:59.737+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:18:59.783 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:18:59.778+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:18:59.877 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:18:59.872+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:00.327 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:00.320+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:00.481 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:00.473+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:00.591 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:00.587+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:00.706 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:00.700+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:00.764 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:00.761+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:00.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:00.786+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:00.920 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:00.912+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:01.295 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:01.291+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:01.460 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:01.454+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:01.606 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:01.602+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:01.730 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:01.725+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:01.730 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:01.725+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:01.761 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:01.756+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:01.953 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:01.948+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:02.295 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:02.290+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:02.504 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:02.500+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:02.632 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:02.627+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:02.754 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:02.750+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:02.768 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:02.762+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:02.776 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:02.772+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:02.933 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:02.928+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:03.262 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:03.258+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:03.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:03.456+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:03.585 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:03.581+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:03.778 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:03.768+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:03.808 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:03.798+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:03.817 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:03.812+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:03.948 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:03.942+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:04.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:04.292+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:04.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:04.423+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:04.596 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:04.592+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:04.609 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:56.458 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.610 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:56.460 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.647 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:57:56.497 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.657 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:56.481 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.659 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:56.484 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.682 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:56.506 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.684 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:56.508 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.697 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:57:56.521 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.751 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:57:56.575 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:19:04.793 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:04.789+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:04.802 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:04.798+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:04.835 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:04.830+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:04.914 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:04.909+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:05.283 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:05.280+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:05.398 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:05.393+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:05.582 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:05.578+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:05.799 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:05.791+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:05.804 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:05.799+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:05.828 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:05.822+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:05.898 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:05.892+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:06.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:06.240+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:06.370 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:06.365+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:06.578 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:06.573+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:06.772 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:06.768+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:06.815 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:06.811+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:06.871 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:06.866+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:06.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:06.884+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:07.236 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:07.231+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:07.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:07.389+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:07.619 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:07.615+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:07.778 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:07.773+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:07.840 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:07.835+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:07.906 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:07.899+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:07.916 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:07.912+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:08.261 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:08.256+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:08.359 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:08.353+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:08.613 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:08.608+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:08.793 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:08.787+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:08.873 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:08.869+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:08.883 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:08.880+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:08.887 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:08.884+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:09.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:09.276+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:09.331 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:09.326+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:09.629 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:09.625+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:09.825 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:09.820+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:09.842 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:09.837+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:09.863 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:09.858+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:09.902 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:09.897+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:10.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:10.307+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:10.316 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:10.311+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:10.607 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:10.602+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:10.821 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:10.817+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:10.841 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:10.835+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:10.858 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:10.853+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:10.903 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:10.896+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:11.280 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:11.275+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:11.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:11.291+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:11.642 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:11.638+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:11.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:11.791+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:11.869 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:11.862+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:11.900 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:11.897+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:11.901 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:11.895+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:12.239 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:12.233+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:12.327 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:12.323+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:12.597 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:12.593+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:12.816 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:12.812+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:12.841 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:12.829+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:12.863 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:12.858+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:12.922 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:12.917+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:13.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:13.282+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:13.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:13.356+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:13.612 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:13.607+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:13.799 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:13.795+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:19:13.831 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:19:13.826+0000 1694f700 -1 osd.7 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.426823+0000 front 2019-12-09T19:17:58.425092+0000 (oldest deadline 2019-12-09T19:18:41.265482+0000) 2019-12-09T19:19:13.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:19:13.827+0000 1694f700 -1 osd.5 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.630606+0000 front 2019-12-09T19:17:57.576299+0000 (oldest deadline 2019-12-09T19:18:39.713075+0000) 2019-12-09T19:19:13.899 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:19:13.895+0000 1694f700 -1 osd.0 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.424625+0000 front 2019-12-09T19:17:58.426112+0000 (oldest deadline 2019-12-09T19:18:46.915767+0000) 2019-12-09T19:19:14.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:19:14.273+0000 1694f700 -1 osd.6 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:57.526133+0000 front 2019-12-09T19:17:57.083487+0000 (oldest deadline 2019-12-09T19:18:39.426914+0000) 2019-12-09T19:19:14.369 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:19:14.365+0000 1694f700 -1 osd.2 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.903724+0000 front 2019-12-09T19:17:58.909940+0000 (oldest deadline 2019-12-09T19:18:44.782729+0000) 2019-12-09T19:19:14.598 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:19:14.589+0000 1694f700 -1 osd.3 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.142361+0000 front 2019-12-09T19:17:58.422997+0000 (oldest deadline 2019-12-09T19:18:40.513359+0000) 2019-12-09T19:19:14.653 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:19:14.646+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e297 v297) 2019-12-09T19:19:14.813 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:19:14.809+0000 1694f700 -1 osd.1 297 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:17:58.422593+0000 front 2019-12-09T19:17:57.809937+0000 (oldest deadline 2019-12-09T19:18:39.355050+0000) 2019-12-09T19:20:11.884 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:11.878+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:12.139 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:12.136+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:12.932 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:12.928+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:13.192 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:13.187+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:13.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:13.568+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:13.964 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:13.955+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:14.224 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:14.219+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:14.564 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:14.558+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:14.989 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:14.985+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:15.275 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:15.271+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:15.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:15.599+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:15.869 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:15.861+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:15.922 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:15.917+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:15.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:15.969+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:16.301 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:16.297+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:16.638 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:16.632+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:16.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:16.806+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:16.876 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:16.871+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:16.919 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:16.912+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:16.955 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:16.951+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:17.293 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:17.289+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:17.609 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:17.603+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:17.836 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:17.832+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:17.839 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:17.835+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:17.931 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:17.927+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:18.005 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:18.001+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:18.329 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:18.324+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:18.652 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:18.647+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:18.838 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:18.833+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:18.856 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:18.851+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:18.948 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:18.943+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:19.025 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:19.020+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:19.046 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:19.044+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:19.289 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:19.284+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:19.607 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:19.602+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:19.842 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:19.836+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:19.867 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:19.862+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:19.958 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:19.952+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:20.002 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:19.998+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:20.029 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:20.023+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:20.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:20.309+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:20.566 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:20.561+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:20.858 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:20.851+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:20.893 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:20.881+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:20.980 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:20.974+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:20.986 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:20.981+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:21.034 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:21.028+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:21.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:21.356+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:21.583 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:21.578+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:21.884 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:21.879+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:21.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:21.927+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:21.987 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:21.981+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:21.990 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:21.985+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:22.036 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:22.030+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:22.334 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:22.329+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:22.606 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:22.601+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:22.866 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:22.862+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:22.946 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:22.940+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:22.960 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:22.954+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:23.001 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:22.997+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:23.027 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:23.021+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:23.365 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:23.362+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:23.579 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:23.573+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:23.886 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:23.882+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:23.946 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:23.939+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:23.970 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:23.965+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:24.024 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:24.020+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:24.074 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:24.070+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:24.385 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:24.380+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:24.583 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:24.577+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:24.915 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:24.910+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:24.967 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:24.962+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:25.015 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:25.011+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:25.016 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:25.010+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:25.063 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:25.058+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:25.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:25.411+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:25.554 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:25.549+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:25.968 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:25.962+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:25.996 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:25.991+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:25.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:25.992+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:26.035 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:26.029+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:26.110 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:26.105+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:26.404 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:26.399+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:26.518 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:26.513+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:26.973 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:26.968+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:27.001 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:26.995+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:27.029 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:27.026+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:27.071 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:27.065+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:27.085 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:27.079+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:27.087 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:27.082+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:27.393 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:27.389+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:27.542 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:27.536+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:27.974 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:27.971+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:27.990 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:27.984+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:27.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:27.992+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:28.077 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:28.072+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:28.093 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:28.087+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:28.095 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:28.089+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:28.444 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:28.439+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:28.501 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:28.496+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:28.936 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:28.931+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:28.950 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:28.944+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:28.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:28.967+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:29.045 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:29.039+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:29.084 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:29.076+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:29.092 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:29.084+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:29.442 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:29.438+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:29.526 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:29.520+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:29.923 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:29.917+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:29.983 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:29.979+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:29.992 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:29.985+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:30.060 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:30.055+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:30.142 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:30.136+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:30.143 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:30.138+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:30.414 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:30.410+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:30.465 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:59:22.314 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.478 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:59:22.327 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.495 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:59:22.319 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.497 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:59:22.322 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.499 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:59:22.324 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.512 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:30.507+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:30.515 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:59:22.340 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.599 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:07:59:22.423 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.644 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:07:59:22.469 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.698 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:07:59:22.547 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:20:30.905 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:30.900+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:30.945 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:30.940+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:31.017 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:31.012+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:31.072 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:31.067+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:31.131 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:31.126+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:31.135 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:31.131+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:31.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:31.443+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:31.564 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:31.559+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:31.908 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:31.904+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:31.908 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:31.902+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:32.045 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:32.039+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:32.057 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:32.053+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:32.107 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:32.101+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:32.112 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:32.107+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:32.491 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:32.487+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:32.607 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:32.598+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:32.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:32.861+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:32.923 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:32.918+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:33.038 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:33.033+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:33.081 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:33.076+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:33.151 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:33.144+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:33.151 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:33.146+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:33.449 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:33.443+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:33.635 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:33.630+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:33.876 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:33.870+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:33.945 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:33.942+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:34.047 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:34.043+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:34.072 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:34.065+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:34.145 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:34.139+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:34.146 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:34.141+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:34.438 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:34.434+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:34.628 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:34.622+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:34.886 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:34.880+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:34.957 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:34.952+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:35.027 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:35.023+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:35.122 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:35.116+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:35.141 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:35.134+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:35.145 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:35.136+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:35.455 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:35.452+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:35.683 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:35.677+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:35.877 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:35.871+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:35.921 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:35.917+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:36.078 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:36.074+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:36.154 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:36.141+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:36.154 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:36.143+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:36.154 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:36.143+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:36.467 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:36.462+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:36.715 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:36.709+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:36.879 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:36.873+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:36.888 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:36.884+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:37.056 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:37.052+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:37.115 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:37.108+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:37.116 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:37.111+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:37.172 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:37.165+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:37.426 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:37.420+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:37.761 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:37.755+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:37.864 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:37.857+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:37.897 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:37.893+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:38.104 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:38.100+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:38.155 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:38.149+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:38.159 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:38.155+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:38.167 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:38.161+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:38.455 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:38.448+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:38.790 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:38.785+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:38.817 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:38.812+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:38.943 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:38.939+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:39.073 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:39.068+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:39.141 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:39.134+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:39.143 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:39.137+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:39.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:39.194+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:39.440 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:39.436+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:39.770 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:39.764+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:39.809 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:39.804+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:39.933 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:39.930+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:40.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:40.105+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:40.131 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:40.124+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:40.132 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:40.127+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:40.165 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:40.159+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:40.426 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:40.421+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:40.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:40.782+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:40.858 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:40.853+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:40.900 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:40.896+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:41.093 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:41.089+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:41.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:41.155+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:41.170 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:41.157+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:41.170 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:41.160+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:41.390 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:41.386+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:41.776 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:41.770+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:41.840 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:41.831+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:41.946 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:41.942+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:42.143 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:42.139+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:42.178 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:42.171+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:42.179 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:42.174+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:42.200 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:42.193+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:42.407 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:42.402+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:42.754 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:42.748+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:42.859 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:42.854+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:42.941 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:42.937+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:43.121 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:43.117+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:43.220 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:43.212+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:43.222 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:43.216+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:43.229 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:43.224+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:43.399 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:43.394+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:43.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:43.781+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:43.887 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:43.881+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:43.984 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:43.979+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:44.162 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:44.158+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:44.188 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:44.182+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:44.209 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:44.197+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:44.209 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:44.199+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:44.354 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:44.349+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:44.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:44.780+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:44.853 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:44.847+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:45.030 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:45.024+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:45.192 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:45.188+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:45.201 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:45.196+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:45.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:45.200+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:45.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:45.234+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:45.344 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:45.339+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:45.368 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:20:45.363+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 40sec e303 v303) 2019-12-09T19:20:45.812 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:45.806+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:45.833 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:45.828+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:46.022 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:46.017+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:46.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:46.167+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:46.204 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:46.197+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:46.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:46.200+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:46.268 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:46.262+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:46.397 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:46.393+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:46.842 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:46.836+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:46.862 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:46.856+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:47.012 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:47.007+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:47.149 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:47.144+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:47.211 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:47.206+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:47.215 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:47.211+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:47.277 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:47.272+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:47.400 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:47.395+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:47.881 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:47.875+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:47.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:47.884+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:48.042 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:48.038+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:48.198 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:48.194+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:48.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:48.195+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:48.204 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:48.199+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:48.308 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:48.302+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:48.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:48.443+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:48.873 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:48.867+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:48.885 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:48.880+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:49.075 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:49.071+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:49.167 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:49.162+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:49.181 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:49.175+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:49.183 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:49.177+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:49.342 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:49.336+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:49.442 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:49.433+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:49.880 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:49.875+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:49.897 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:49.891+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:50.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:50.067+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:50.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:50.124+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:50.210 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:50.203+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:50.210 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:50.206+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:50.344 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:50.339+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:50.385 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:20:50.376+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 40sec e303 v303) 2019-12-09T19:20:50.488 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:50.484+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:50.864 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:50.858+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:50.918 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:50.913+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:51.046 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:51.041+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:51.114 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:51.109+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:51.239 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:51.234+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:51.243 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:51.239+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:51.376 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:51.371+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:51.491 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:51.485+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:51.908 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:51.896+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:51.914 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:51.908+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:52.013 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:52.008+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:52.090 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:52.085+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:52.214 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:52.209+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:52.218 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:52.214+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:52.382 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:52.376+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:52.522 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:52.516+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:52.878 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:52.872+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:52.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:52.922+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:53.048 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:53.044+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:53.085 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:53.082+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:53.223 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:53.216+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:53.223 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:53.218+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:53.376 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:53.371+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:53.556 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:53.553+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:53.914 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:53.908+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:53.923 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:53.917+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:54.040 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:54.036+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:54.066 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:54.061+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:54.184 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:54.179+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:54.188 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:54.184+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:54.387 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:54.382+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:54.564 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:54.561+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:54.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:54.866+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:54.939 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:54.934+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:55.062 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:55.057+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:55.112 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:55.107+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:55.204 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:55.198+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:55.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:55.200+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:55.360 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:55.356+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:55.396 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:20:55.392+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 40sec e303 v303) 2019-12-09T19:20:55.525 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:55.521+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:55.860 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:55.856+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:55.933 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:55.927+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:56.025 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:56.020+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:56.165 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:56.156+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:56.176 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:56.170+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:56.177 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:56.172+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:56.334 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:56.330+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:56.488 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:56.484+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:56.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:56.891+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:56.911 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:56.905+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:57.011 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:57.007+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:57.174 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:57.170+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:57.216 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:57.211+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:57.219 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:57.215+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:57.385 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:57.379+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:57.461 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:57.456+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:57.864 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:57.858+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:57.951 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:57.944+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:58.013 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:58.009+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:58.132 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:58.127+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:58.205 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:58.199+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:58.207 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:58.201+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:58.400 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:58.395+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:58.469 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:58.463+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:58.895 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:58.889+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:58.964 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:58.956+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:20:59.020 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:20:59.015+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:20:59.124 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:20:59.118+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:20:59.223 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:59.218+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:20:59.227 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:20:59.223+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:20:59.373 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:20:59.368+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:20:59.461 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:20:59.456+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:20:59.948 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:20:59.944+0000 1694f700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.920432+0000 front 2019-12-09T19:19:30.294882+0000 (oldest deadline 2019-12-09T19:20:13.415226+0000) 2019-12-09T19:20:59.998 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:20:59.994+0000 1694f700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:29.952428+0000 front 2019-12-09T19:19:30.294125+0000 (oldest deadline 2019-12-09T19:20:15.837382+0000) 2019-12-09T19:21:00.075 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:21:00.069+0000 1694f700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.639274+0000 front 2019-12-09T19:19:31.049594+0000 (oldest deadline 2019-12-09T19:20:15.892257+0000) 2019-12-09T19:21:00.161 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:21:00.155+0000 1694f700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.331246+0000 front 2019-12-09T19:19:31.328981+0000 (oldest deadline 2019-12-09T19:20:11.716319+0000) 2019-12-09T19:21:00.212 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:21:00.206+0000 1694f700 -1 osd.4 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:19:46.166998+0000 (oldest deadline 2019-12-09T19:20:26.166998+0000) 2019-12-09T19:21:00.213 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:21:00.209+0000 1694f700 -1 osd.4 303 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e301) 2019-12-09T19:21:00.375 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:21:00.371+0000 1694f700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:30.598714+0000 front 2019-12-09T19:19:31.046996+0000 (oldest deadline 2019-12-09T19:20:15.212585+0000) 2019-12-09T19:21:00.492 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:21:00.483+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 40sec e303 v303) 2019-12-09T19:21:00.494 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:21:00.487+0000 1694f700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:19:31.518172+0000 front 2019-12-09T19:19:31.469637+0000 (oldest deadline 2019-12-09T19:20:11.965690+0000) 2019-12-09T19:21:29.281 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:21:29.253+0000 11caf700 -1 osd.0 307 build_incremental_map_msg missing incremental map 307 2019-12-09T19:21:29.461 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:21:29.370+0000 3578e700 -1 osd.0 307 build_incremental_map_msg missing incremental map 307 2019-12-09T19:21:29.685 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:21:29.592+0000 38794700 -1 osd.5 307 build_incremental_map_msg missing incremental map 307 2019-12-09T19:21:29.715 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:21:29.508+0000 36f91700 -1 osd.0 307 build_incremental_map_msg missing incremental map 307 2019-12-09T19:21:31.938 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:21:31.780+0000 31f87700 -1 osd.3 308 build_incremental_map_msg missing incremental map 308 2019-12-09T19:21:32.020 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:21:31.842+0000 36790700 -1 osd.3 308 build_incremental_map_msg missing incremental map 308 2019-12-09T19:21:32.044 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:21:31.945+0000 36f91700 -1 osd.3 308 build_incremental_map_msg missing incremental map 308 2019-12-09T19:21:32.817 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:21:32.681+0000 37792700 -1 osd.3 308 build_incremental_map_msg missing incremental map 308 2019-12-09T19:21:34.614 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:21:34.578+0000 33f8b700 -1 osd.3 309 build_incremental_map_msg missing incremental map 308 2019-12-09T19:22:32.699 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:32.693+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:33.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:33.304+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:33.555 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:33.546+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:33.651 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:33.645+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:33.689 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:33.683+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:34.187 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:34.182+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:34.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:34.342+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:34.382 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:34.376+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:34.428 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:34.423+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:34.534 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:34.527+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:34.643 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:34.637+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:34.728 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:34.721+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:35.171 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:35.166+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:35.377 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:35.371+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:35.409 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:35.405+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:35.474 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:35.469+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:35.567 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:35.562+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:35.674 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:35.668+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:35.747 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:35.741+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:36.141 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:36.135+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:36.346 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:36.341+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:36.374 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:36.368+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:36.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:36.466+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:36.545 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:36.540+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:36.693 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:36.688+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:36.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:36.782+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:37.199 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:37.194+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:37.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:37.303+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:37.353 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:37.349+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:37.493 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:37.488+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:37.517 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:37.513+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:37.730 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:37.724+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:37.752 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:37.747+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:38.207 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:38.203+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:38.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:38.306+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:38.355 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:38.350+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:38.473 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:38.464+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:38.512 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:38.508+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:38.740 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:38.737+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:38.784 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:38.780+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:39.192 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:39.186+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:39.321 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:39.316+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:39.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:39.333+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:39.436 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:39.430+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:39.488 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:39.484+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:39.754 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:39.749+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:39.783 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:39.780+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:40.230 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:40.226+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:40.305 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:40.301+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:40.352 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:40.343+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:40.482 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:40.477+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:40.485 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:40.479+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:40.713 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:40.708+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:40.789 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:40.784+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:41.262 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:41.258+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:41.284 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:41.279+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:41.372 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:41.368+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:41.517 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:41.509+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:41.525 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:41.519+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:41.746 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:41.741+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:41.775 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:41.769+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:42.220 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:42.214+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:42.287 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:42.282+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:42.335 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:42.330+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:42.508 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:42.503+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:42.522 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:42.517+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:42.613 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:01:34.463 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.626 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:01:34.450 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.629 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:01:34.453 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.629 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:01:34.477 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.634 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:01:34.458 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.634 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:01:34.459 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.714 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:42.710+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:42.788 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:42.784+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:42.789 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:01:34.612 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.789 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:01:34.637 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:42.791 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:01:34.616 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:22:43.203 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:43.199+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:43.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:43.319+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:43.351 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:43.347+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:43.494 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:43.489+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:43.551 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:43.545+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:43.746 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:43.740+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:43.758 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:43.754+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:44.162 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:44.158+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:44.305 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:44.301+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:44.307 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:44.304+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:44.486 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:44.481+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:44.573 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:44.568+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:44.763 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:44.758+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:44.765 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:44.761+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:45.173 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:45.168+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:45.265 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:45.260+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:45.292 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:45.288+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:45.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:45.442+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:45.567 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:45.562+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:45.777 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:45.773+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:45.785 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:45.781+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:46.220 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:46.216+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:46.287 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:46.281+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:46.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:46.293+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:46.476 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:46.471+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:46.575 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:46.571+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:46.774 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:46.769+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:46.827 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:46.822+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:47.243 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:47.238+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:47.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:47.297+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:47.351 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:47.346+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:47.510 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:47.504+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:47.553 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:47.547+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:47.814 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:47.808+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:47.817 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:47.812+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:48.225 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:48.220+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:48.340 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:48.335+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:48.350 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:48.346+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:48.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:48.471+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:48.542 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:48.537+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:48.850 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:48.844+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:48.860 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:48.856+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:49.273 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:49.268+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:49.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:49.310+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:49.324 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:49.321+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:49.519 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:49.514+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:49.537 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:49.533+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:49.840 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:49.835+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:49.850 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:49.846+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:50.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:50.277+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:50.320 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:50.316+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:50.332 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:50.328+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:50.494 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:50.489+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:50.570 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:50.565+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:50.818 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:50.813+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:50.844 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:50.840+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:51.305 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:51.300+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:51.336 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:51.331+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:51.358 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:51.354+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:51.508 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:51.503+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:51.580 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:51.575+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:51.816 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:51.812+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:51.816 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:51.811+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:52.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:52.310+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:52.328 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:52.324+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:52.349 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:52.345+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:52.470 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:52.465+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:52.609 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:52.604+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:52.821 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:52.815+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:52.902 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:52.863+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:53.316 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:53.311+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:53.340 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:53.336+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:53.363 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:53.357+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:53.443 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:53.438+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:53.573 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:53.568+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:53.844 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:53.838+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:53.916 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:53.910+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:54.330 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:54.326+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:54.358 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:54.355+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:54.361 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:54.358+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:54.483 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:54.478+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:54.605 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:54.600+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:54.838 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:54.832+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:54.938 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:54.933+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:55.283 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:55.279+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:55.366 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:55.362+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:55.387 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:55.383+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:55.456 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:55.451+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:55.626 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:55.620+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:55.874 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:55.870+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:55.961 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:55.955+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:56.337 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:56.332+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:56.385 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:56.381+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:56.426 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:56.421+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:56.433 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:56.427+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:56.674 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:56.668+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:56.852 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:56.847+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:56.991 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:56.985+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:57.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:57.342+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:57.384 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:57.381+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:57.408 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:57.404+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:57.410 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:57.405+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:57.660 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:57.655+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:57.824 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:57.820+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:58.025 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:58.020+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:58.317 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:58.310+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:58.341 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:58.336+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:58.438 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:58.433+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:58.438 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:58.434+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:58.711 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:58.705+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:58.800 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:58.795+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:22:59.011 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:22:59.004+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:22:59.272 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:22:59.268+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:22:59.392 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:22:59.388+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:22:59.395 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:22:59.391+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:22:59.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:22:59.396+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:22:59.754 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:22:59.749+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:22:59.828 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:22:59.824+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:00.040 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:00.035+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:00.313 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:00.308+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:00.358 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:00.354+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:00.438 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:00.434+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:00.445 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:00.439+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:00.726 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:00.722+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:00.855 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:00.850+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:01.059 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:01.053+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:01.299 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:01.294+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:01.355 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:01.350+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:01.457 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:01.452+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:01.475 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:01.469+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:01.690 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:01.686+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:01.814 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:01.809+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:02.050 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:02.044+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:02.307 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:02.302+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:02.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:02.400+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:02.440 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:02.435+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:02.461 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:02.456+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:02.646 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:02.642+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:02.805 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:02.800+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:03.018 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:03.013+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:03.278 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:03.273+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:03.452 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:03.446+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:03.461 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:03.453+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:03.486 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:03.481+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:03.622 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:03.616+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:03.800 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:03.796+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:04.060 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:04.056+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:04.292 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:04.288+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:04.430 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:04.424+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:04.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:04.433+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:04.497 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:04.492+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:04.608 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:04.602+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:04.851 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:04.846+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:05.074 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:05.069+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:05.261 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:05.256+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:05.399 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:05.393+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:05.445 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:05.440+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:05.511 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:05.507+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:05.610 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:05.606+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:05.835 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:05.831+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:06.035 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:06.030+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:06.041 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:23:06.038+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e309 v309) 2019-12-09T19:23:06.303 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:06.299+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:06.402 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:06.396+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:06.411 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:06.407+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:06.539 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:06.534+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:06.646 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:06.643+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:06.805 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:06.800+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:07.090 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:07.086+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:07.316 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:07.312+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:07.415 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:07.410+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:07.455 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:07.451+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:07.567 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:07.562+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:07.688 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:07.683+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:07.857 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:07.851+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:08.099 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:08.092+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:08.333 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:08.328+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:08.460 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:08.456+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:08.475 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:08.471+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:08.524 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:08.520+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:08.734 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:08.728+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:08.891 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:08.886+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:09.115 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:09.109+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:09.353 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:09.349+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:09.506 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:09.502+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:09.510 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:09.504+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:09.526 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:09.523+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:09.696 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:09.691+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:09.866 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:09.862+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:10.163 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:23:10.159+0000 1694f700 -1 osd.7 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.717421+0000 front 2019-12-09T19:21:47.710896+0000 (oldest deadline 2019-12-09T19:22:31.796568+0000) 2019-12-09T19:23:10.355 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:23:10.351+0000 1694f700 -1 osd.1 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:48.946285+0000 front 2019-12-09T19:21:49.194325+0000 (oldest deadline 2019-12-09T19:22:33.026448+0000) 2019-12-09T19:23:10.509 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:23:10.502+0000 1694f700 -1 osd.5 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.980436+0000 front 2019-12-09T19:21:49.769990+0000 (oldest deadline 2019-12-09T19:22:33.751864+0000) 2019-12-09T19:23:10.535 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:23:10.530+0000 1694f700 -1 osd.0 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:50.559021+0000 front 2019-12-09T19:21:52.132009+0000 (oldest deadline 2019-12-09T19:22:33.691280+0000) 2019-12-09T19:23:10.564 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:23:10.560+0000 1694f700 -1 osd.2 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.403595+0000 front 2019-12-09T19:21:49.218176+0000 (oldest deadline 2019-12-09T19:22:33.667201+0000) 2019-12-09T19:23:10.655 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:23:10.650+0000 1694f700 -1 osd.6 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:47.381909+0000 front 2019-12-09T19:21:47.403052+0000 (oldest deadline 2019-12-09T19:22:33.141615+0000) 2019-12-09T19:23:10.895 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:23:10.891+0000 1694f700 -1 osd.3 309 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:21:49.842551+0000 front 2019-12-09T19:21:49.787845+0000 (oldest deadline 2019-12-09T19:22:33.583996+0000) 2019-12-09T19:23:11.118 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:23:11.113+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 43sec e309 v309) 2019-12-09T19:23:49.439 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.426+0000 34f8d700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:49.470 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.437+0000 34f8d700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:49.475 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.450+0000 3578e700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:49.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.465+0000 3578e700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:49.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.492+0000 31f87700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:49.636 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:49.540+0000 36f91700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:50.327 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:50.321+0000 32f89700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:23:50.419 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:23:50.407+0000 36f91700 -1 osd.4 314 build_incremental_map_msg missing incremental map 314 2019-12-09T19:24:08.471 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:08.466+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:09.503 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:09.498+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:10.526 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:10.519+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:10.937 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:10.933+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:11.513 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:11.510+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:11.748 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:11.743+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:11.897 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:11.892+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:11.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:11.952+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:12.284 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:12.278+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:12.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:12.469+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:12.706 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:12.700+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:12.892 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:12.887+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:12.950 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:12.944+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:12.990 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:12.986+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:13.284 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:13.279+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:13.485 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:13.481+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:13.757 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:13.751+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:13.874 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:13.867+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:13.966 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:13.960+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:13.967 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:13.961+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:14.316 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:14.310+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:14.467 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:14.463+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:14.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:14.750+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:14.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:14.900+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:14.918 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:14.915+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:14.935 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:14.929+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:15.342 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:15.337+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:15.512 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:15.506+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:15.767 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:15.762+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:15.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:15.879+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:15.921 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:15.916+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:15.936 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:15.931+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:16.368 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:16.362+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:16.556 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:16.552+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:16.764 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:16.758+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:16.858 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:16.854+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:16.913 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:16.910+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:16.965 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:16.959+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:17.408 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:17.402+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:17.596 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:17.593+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:17.735 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:17.730+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:17.894 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:17.886+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:17.947 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:17.941+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:17.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:17.958+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:18.401 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:18.395+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:18.624 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:18.620+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:18.739 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:18.733+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:18.913 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:18.909+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:18.966 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:18.962+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:18.969 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:18.964+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:19.451 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:19.446+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:19.661 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:19.656+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:19.743 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:19.738+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:19.931 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:19.926+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:19.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:19.934+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:20.019 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:20.015+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:20.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:20.423+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:20.541 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:20.528+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:20.634 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:20.627+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:20.727 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:20.721+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:20.949 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:20.944+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:20.950 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:20.944+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:20.974 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:20.970+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:21.445 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:21.440+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:21.524 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:21.518+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:21.603 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:21.599+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:21.734 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:21.728+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:21.941 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:21.937+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:22.000 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:21.994+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:22.023 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:22.019+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:22.433 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:22.412+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:22.483 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:22.477+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:22.575 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:22.570+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:22.736 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:22.731+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:22.992 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:22.988+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:23.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:23.002+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:23.039 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:23.034+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:23.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:23.440+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:23.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:23.501+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:23.539 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:23.534+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:23.749 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:23.743+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:24.019 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:24.015+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:24.020 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:24.016+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:24.024 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:24.018+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:24.499 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:24.494+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:24.537 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:24.529+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:24.589 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:24.585+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:24.759 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:24.754+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:24.995 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:24.990+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:25.004 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:25.000+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:25.070 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:25.066+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:25.492 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:25.486+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:25.512 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:25.507+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:25.643 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:25.639+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:25.775 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:25.769+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:25.979 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:25.973+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:25.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:25.977+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:26.083 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:26.079+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:26.489 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:26.483+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:26.523 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:26.518+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:26.528 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:26.523+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:26.668 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:26.664+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:26.790 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:26.785+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:26.985 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:26.975+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:27.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:27.002+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:27.119 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:27.115+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:27.496 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:27.489+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:27.497 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:27.491+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:27.499 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:27.493+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:27.704 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:27.700+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:27.819 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:27.813+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:27.945 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:27.941+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:28.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:28.038+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:28.161 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:28.157+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:28.500 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:28.494+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:28.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:28.500+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:28.532 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:28.528+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:28.722 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:28.717+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:28.813 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:28.807+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:28.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:28.899+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:29.029 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:29.024+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:29.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:29.133+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:29.522 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:29.516+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:29.522 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:29.519+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:29.575 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:29.571+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:29.713 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:29.709+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:29.792 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:29.785+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:29.858 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:29.854+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:30.068 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:30.064+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:30.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:30.094+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:30.548 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:30.542+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:30.550 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:30.545+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:30.567 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:30.562+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:30.692 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:30.686+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:30.760 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:30.756+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:30.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:30.873+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:31.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:31.093+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:31.102 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:31.098+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:31.560 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:31.552+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:31.560 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:31.555+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:31.608 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:31.604+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:31.701 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:31.697+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:31.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:31.751+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:31.916 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:31.913+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:32.056 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:32.053+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:32.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:32.077+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:32.591 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:32.584+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:32.592 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:32.587+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:32.620 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:32.614+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:32.678 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:32.674+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:32.815 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:32.803+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:32.936 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:32.930+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:33.011 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:33.008+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:33.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:33.077+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:33.602 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:33.595+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:33.627 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:33.622+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:33.673 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:33.667+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:33.716 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:33.711+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:33.834 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:33.828+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:33.891 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:33.888+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:34.059 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:34.055+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:34.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:34.076+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:34.601 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:34.596+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:34.604 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:34.600+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:34.627 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:34.622+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:34.763 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:34.759+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:34.864 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:34.860+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:34.884 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:34.879+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:35.028 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:35.024+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:35.040 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:35.035+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:35.592 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:35.587+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:35.606 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:35.600+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:35.607 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:35.603+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:35.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:35.790+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:35.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:35.868+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:35.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:35.899+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:36.036 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:36.025+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:36.092 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:36.087+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:36.645 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:36.639+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:36.664 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:36.653+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:36.664 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:36.656+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:36.786 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:36.783+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:36.894 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:36.889+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:36.913 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:36.910+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:37.024 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:37.019+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:37.084 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:37.077+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:37.603 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:37.596+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:37.668 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:37.661+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:37.669 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:37.664+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:37.770 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:37.765+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:37.866 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:37.860+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:37.929 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:37.924+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:38.031 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:38.027+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:38.060 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:38.055+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:38.612 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:38.607+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:38.635 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:38.630+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:38.637 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:38.634+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:38.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:38.786+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:38.847 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:38.837+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:38.974 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:38.969+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:39.000 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:38.996+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:39.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:39.051+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:39.599 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:39.594+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:39.622 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:39.617+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:39.627 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:39.623+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:39.767 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:39.763+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:39.886 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:39.881+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:39.953 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:39.946+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:39.960 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:39.955+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:40.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:40.052+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:40.584 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:40.579+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:40.639 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:40.634+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:40.644 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:40.639+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:40.810 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:40.807+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:40.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:40.909+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:40.917 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:40.911+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:40.960 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:40.954+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:41.101 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:41.095+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:41.448 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:24:41.439+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e315 v315) 2019-12-09T19:24:41.570 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:41.565+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:41.654 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:41.647+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:41.655 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:41.650+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:41.777 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:41.771+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:41.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:41.902+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:41.950 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:41.944+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:41.958 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:41.951+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:42.142 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:42.137+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:42.531 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:42.525+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:42.697 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:42.690+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:42.698 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:42.693+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:42.810 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:42.804+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:42.922 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:42.917+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:42.952 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:42.947+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:42.958 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:42.954+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:43.180 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:43.175+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:43.581 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:43.576+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:43.750 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:43.744+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:43.751 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:43.746+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:43.803 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:43.797+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:43.921 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:43.905+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:43.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:43.916+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:43.921 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:43.899+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:44.139 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:44.134+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:44.629 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:44.623+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:44.721 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:44.716+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:44.722 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:44.718+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:44.832 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:44.827+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:44.890 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:44.885+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:44.932 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:44.927+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:44.948 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:44.943+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:45.160 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:45.155+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:45.668 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:45.663+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:45.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:45.759+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:45.769 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:45.764+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:45.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:45.789+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:45.880 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:45.875+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:45.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:45.883+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:45.982 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:45.978+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:46.216 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:46.207+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:46.508 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:24:46.504+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e315 v315) 2019-12-09T19:24:46.637 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:46.632+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:46.757 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:46.752+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:46.760 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:46.756+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:46.837 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:46.832+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:46.841 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:46.836+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:46.874 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:46.869+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:46.955 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:46.950+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:47.185 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:47.179+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:47.672 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:47.666+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:47.739 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:47.734+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:47.743 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:47.739+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:47.825 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:47.815+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:47.838 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:47.832+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:47.864 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:47.859+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:47.959 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:47.954+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:48.193 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:48.188+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:48.666 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:48.661+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:48.775 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:48.769+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:48.783 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:48.772+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:48.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:48.784+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:48.850 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:48.845+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:48.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:48.877+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:48.917 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:48.912+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:49.169 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:49.163+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:49.722 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:49.714+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:49.800 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:49.794+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:49.825 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:49.819+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:49.827 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:49.821+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:49.873 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:49.869+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:49.880 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:49.875+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:49.896 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:49.890+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:50.147 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:50.141+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:50.770 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:50.758+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:50.794 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:50.789+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:50.797 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:50.793+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:50.812 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:50.807+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:50.878 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:50.873+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:50.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:50.883+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:50.895 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:50.891+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:51.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:51.156+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:51.533 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:24:51.526+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e315 v315) 2019-12-09T19:24:51.774 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:51.767+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:51.775 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:51.770+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:51.784 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:51.778+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:51.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:51.785+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:51.882 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:51.876+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:51.933 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:51.929+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:51.949 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:51.944+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:52.156 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:52.150+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:52.770 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:52.763+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:52.771 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:52.766+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:52.775 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:52.769+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:52.820 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:52.814+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:52.875 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:52.870+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:52.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:52.957+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:52.989 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:52.983+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:53.127 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:53.121+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:53.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:53.760+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:53.800 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:53.795+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:53.802 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:53.795+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:53.804 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:53.798+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:53.837 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:53.832+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:53.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:53.933+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:54.008 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:54.002+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:54.085 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:54.078+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:54.820 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:54.814+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:54.821 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:54.814+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:54.833 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:54.827+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:54.835 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:54.830+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:54.870 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:54.864+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:54.904 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:54.900+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:55.009 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:55.003+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:55.092 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:55.087+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:55.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:55.810+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:55.820 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:55.815+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:55.842 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:55.836+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:55.870 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:55.864+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:55.908 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:55.902+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:55.912 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:55.907+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:55.986 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:55.979+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:56.116 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:56.111+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:56.547 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:24:56.541+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e315 v315) 2019-12-09T19:24:56.845 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:56.840+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:56.845 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:56.841+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:56.854 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:56.850+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:56.862 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:56.857+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:56.862 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:56.857+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:56.897 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:56.892+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:56.958 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:56.953+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:57.160 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:57.154+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:57.813 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:57.808+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:57.816 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:57.812+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:57.820 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:57.815+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:57.863 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:57.859+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:57.866 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:57.861+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:57.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:57.914+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:57.933 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:57.927+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:58.146 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:58.140+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:58.823 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:58.818+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:58.825 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:58.815+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:58.829 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:58.817+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:58.854 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:58.850+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:58.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:58.867+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:58.917 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:58.911+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:58.984 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:58.978+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:24:59.124 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:24:59.118+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:24:59.839 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:24:59.834+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:24:59.848 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:59.842+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:24:59.849 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:24:59.844+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:24:59.856 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:24:59.849+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:24:59.864 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:24:59.860+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:24:59.941 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:24:59.936+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:24:59.984 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:24:59.978+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:25:00.156 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:25:00.150+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:25:00.795 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:25:00.789+0000 1694f700 -1 osd.1 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:24.985791+0000 front 2019-12-09T19:23:24.977720+0000 (oldest deadline 2019-12-09T19:24:08.422690+0000) 2019-12-09T19:25:00.824 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:25:00.818+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:25:00.833 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:25:00.826+0000 1694f700 -1 osd.4 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:23:45.728995+0000 (oldest deadline 2019-12-09T19:24:25.728995+0000) 2019-12-09T19:25:00.834 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:25:00.829+0000 1694f700 -1 osd.4 315 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:25:00.864 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:00.859+0000 1694f700 -1 osd.6 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.356208+0000 front 2019-12-09T19:23:26.364789+0000 (oldest deadline 2019-12-09T19:24:12.176043+0000) 2019-12-09T19:25:00.942 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:25:00.936+0000 1694f700 -1 osd.7 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:27.333492+0000 front 2019-12-09T19:23:28.508955+0000 (oldest deadline 2019-12-09T19:24:11.335937+0000) 2019-12-09T19:25:00.993 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:25:00.988+0000 1694f700 -1 osd.2 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:26.303098+0000 front 2019-12-09T19:23:26.355711+0000 (oldest deadline 2019-12-09T19:24:12.149533+0000) 2019-12-09T19:25:01.152 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:25:01.147+0000 1694f700 -1 osd.5 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.448226+0000 front 2019-12-09T19:23:28.491557+0000 (oldest deadline 2019-12-09T19:24:11.781895+0000) 2019-12-09T19:25:01.641 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:25:01.626+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e315 v315) 2019-12-09T19:25:01.877 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:25:01.873+0000 1694f700 -1 osd.0 315 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:23:28.038107+0000 front 2019-12-09T19:23:28.029723+0000 (oldest deadline 2019-12-09T19:24:10.750333+0000) 2019-12-09T19:25:02.008 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:25:01.836+0000 1694f700 -1 osd.4 316 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e313) 2019-12-09T19:25:28.330 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:25:28.312+0000 3578e700 -1 osd.0 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.366 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:25:28.353+0000 3638f700 -1 osd.2 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.553 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:25:28.516+0000 3378a700 -1 osd.2 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.561 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:25:28.434+0000 36f91700 -1 osd.0 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.570 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:25:28.463+0000 37f93700 -1 osd.0 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.571 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:28.561+0000 35f8f700 -1 osd.6 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.578 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:25:28.560+0000 38393700 -1 osd.2 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.597 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:28.590+0000 33f8b700 -1 osd.6 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.752 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:28.685+0000 38794700 -1 osd.6 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.802 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:28.722+0000 35f8f700 -1 osd.6 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:28.952 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:25:28.893+0000 32788700 -1 osd.6 319 build_incremental_map_msg missing incremental map 319 2019-12-09T19:25:30.106 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:25:30.031+0000 36f91700 -1 osd.3 320 build_incremental_map_msg missing incremental map 320 2019-12-09T19:25:30.120 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:25:30.038+0000 36f91700 -1 osd.3 320 build_incremental_map_msg missing incremental map 320 2019-12-09T19:25:30.198 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:25:30.094+0000 37792700 -1 osd.3 320 build_incremental_map_msg missing incremental map 320 2019-12-09T19:25:30.720 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:25:30.697+0000 37f93700 -1 osd.3 320 build_incremental_map_msg missing incremental map 320 2019-12-09T19:26:29.152 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:29.147+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:30.014 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:30.007+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:30.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:30.169+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:31.015 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:31.010+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:31.166 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:31.162+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:31.410 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:31.406+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:32.069 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:32.063+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:32.176 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:32.172+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:32.439 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:32.434+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:32.853 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:32.846+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:33.106 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:33.101+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:33.159 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:33.154+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:33.198 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:33.194+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:33.486 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:33.483+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:33.592 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:33.586+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:33.850 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:33.845+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:34.120 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:34.116+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:34.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:34.146+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:34.184 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:34.180+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:34.534 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:34.529+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:34.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:34.545+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:34.849 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:34.845+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:35.172 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:35.167+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:35.186 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:35.181+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:35.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:35.201+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:35.525 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:35.521+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:35.528 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:35.523+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:35.813 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:35.808+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:36.215 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:36.211+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:36.217 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:36.213+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:36.259 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:36.253+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:36.556 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:36.552+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:36.578 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:36.574+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:36.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:36.770+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:37.219 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:37.215+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:37.220 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:37.215+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:37.251 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:37.247+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:37.574 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:37.570+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:37.593 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:37.589+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:37.765 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:37.761+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:38.176 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:38.172+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:38.219 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:38.214+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:38.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:38.293+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:38.589 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:38.585+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:38.606 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:38.602+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:38.814 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:38.809+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:39.214 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:39.210+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:39.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:39.219+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:39.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:39.320+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:39.564 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:39.559+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:39.628 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:39.624+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:39.783 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:39.779+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:40.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:40.181+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:40.257 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:40.253+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:40.317 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:40.313+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:40.550 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:40.544+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:40.592 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:40.587+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:40.772 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:40.768+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:41.148 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:41.143+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:41.275 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:41.271+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:41.340 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:41.336+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:41.517 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:41.512+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:41.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:41.547+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:41.816 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:41.812+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:42.105 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:42.101+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:42.131 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:05:33.981 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.136 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:05:33.985 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.146 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:05:33.971 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.151 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:05:33.975 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.154 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:05:33.978 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.157 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:05:33.982 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.306 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:42.302+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:42.372 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:42.368+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:42.420 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:05:34.245 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.443 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:05:34.293 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.463 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:05:34.287 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:26:42.519 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:42.515+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:42.533 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:42.529+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:42.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:42.827+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:43.095 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:43.090+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:43.346 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:43.342+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:43.368 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:43.364+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:43.539 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:43.535+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:43.560 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:43.555+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:43.840 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:43.835+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:44.103 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:44.099+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:44.335 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:44.331+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:44.394 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:44.390+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:44.504 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:44.499+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:44.578 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:44.572+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:44.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:44.816+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:45.078 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:45.073+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:45.347 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:45.343+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:45.374 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:45.371+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:45.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:45.469+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:45.586 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:45.582+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:45.787 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:45.781+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:46.053 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:46.047+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:46.313 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:46.309+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:46.345 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:46.341+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:46.436 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:46.427+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:46.539 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:46.534+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:46.791 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:46.785+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:47.071 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:47.066+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:47.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:47.292+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:47.388 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:47.384+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:47.464 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:47.460+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:47.589 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:47.585+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:47.771 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:47.765+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:48.030 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:48.025+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:48.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:48.329+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:48.425 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:48.417+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:48.467 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:48.463+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:48.642 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:48.637+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:48.818 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:48.814+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:48.997 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:48.992+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:49.323 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:49.319+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:49.455 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:49.447+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:49.516 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:49.510+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:49.689 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:49.684+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:49.805 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:49.799+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:49.969 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:49.960+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:50.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:50.321+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:50.420 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:50.415+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:50.477 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:50.473+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:50.669 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:50.664+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:50.773 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:50.769+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:50.931 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:50.925+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:51.303 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:51.299+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:51.465 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:51.461+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:51.495 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:51.491+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:51.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:51.644+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:51.770 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:51.766+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:51.921 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:51.911+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:52.299 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:52.290+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:52.476 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:52.471+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:52.503 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:52.498+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:52.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:52.598+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:52.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:52.791+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:52.923 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:52.913+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:53.309 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:53.304+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:53.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:53.472+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:53.477 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:53.473+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:53.650 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:53.644+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:53.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:53.797+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:53.916 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:53.910+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:54.283 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:54.278+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:54.451 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:54.446+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:54.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:54.469+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:54.683 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:54.678+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:54.776 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:54.771+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:54.934 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:54.929+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:55.249 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:55.245+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:55.483 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:55.478+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:55.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:55.504+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:55.676 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:55.671+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:55.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:55.797+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:55.956 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:55.950+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:56.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:56.220+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:56.439 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:56.433+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:56.486 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:56.481+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:56.666 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:56.660+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:56.812 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:56.807+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:56.960 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:56.955+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:57.196 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:57.193+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:57.428 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:57.423+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:57.510 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:57.505+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:57.670 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:57.665+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:57.818 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:57.814+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:57.944 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:57.935+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:58.182 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:58.177+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:58.462 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:58.456+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:58.466 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:58.462+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:58.662 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:58.656+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:58.839 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:58.834+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:58.930 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:58.926+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:26:59.193 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:26:59.189+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:26:59.438 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:26:59.434+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:26:59.455 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:26:59.451+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:26:59.647 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:26:59.641+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:26:59.869 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:26:59.865+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:26:59.953 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:26:59.948+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:00.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:00.149+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:00.407 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:00.402+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:00.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:00.426+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:00.609 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:00.602+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:00.882 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:00.877+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:00.979 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:00.972+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:01.134 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:01.129+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:01.389 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:01.385+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:01.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:01.441+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:01.594 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:01.590+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:01.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:01.834+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:01.942 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:01.936+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:02.054 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:27:02.048+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e320 v320) 2019-12-09T19:27:02.186 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:02.182+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:02.350 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:02.346+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:02.488 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:02.483+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:02.599 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:27:02.594+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.075154+0000 front 2019-12-09T19:25:48.160974+0000 (oldest deadline 2019-12-09T19:27:02.139508+0000) 2019-12-09T19:27:02.614 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:02.607+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:02.808 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:02.803+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:02.923 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:02.918+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:03.202 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:03.199+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:03.327 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:03.322+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:03.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:03.467+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:03.591 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:27:03.587+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.075154+0000 front 2019-12-09T19:25:48.160974+0000 (oldest deadline 2019-12-09T19:27:02.139508+0000) 2019-12-09T19:27:03.604 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:03.599+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:03.792 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:03.788+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:03.973 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:03.969+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:04.224 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:04.220+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:04.329 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:04.325+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:04.461 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:04.457+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:04.588 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:27:04.584+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.075154+0000 front 2019-12-09T19:25:48.160974+0000 (oldest deadline 2019-12-09T19:27:02.139508+0000) 2019-12-09T19:27:04.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:04.608+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:04.785 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:04.780+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:04.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:04.968+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:05.209 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:05.204+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:05.334 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:05.329+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:05.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:05.442+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:05.559 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:27:05.555+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.075154+0000 front 2019-12-09T19:25:48.160974+0000 (oldest deadline 2019-12-09T19:27:02.139508+0000) 2019-12-09T19:27:05.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:05.570+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:05.776 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:05.771+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:05.986 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:05.975+0000 1694f700 -1 osd.6 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.060050+0000 front 2019-12-09T19:25:49.316877+0000 (oldest deadline 2019-12-09T19:26:32.934232+0000) 2019-12-09T19:27:06.183 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:27:06.179+0000 1694f700 -1 osd.0 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.630443+0000 front 2019-12-09T19:25:47.878404+0000 (oldest deadline 2019-12-09T19:26:29.843328+0000) 2019-12-09T19:27:06.365 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:27:06.360+0000 1694f700 -1 osd.1 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.625100+0000 front 2019-12-09T19:25:44.950405+0000 (oldest deadline 2019-12-09T19:26:28.504606+0000) 2019-12-09T19:27:06.460 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:06.455+0000 1694f700 -1 osd.3 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.087068+0000 front 2019-12-09T19:25:45.366716+0000 (oldest deadline 2019-12-09T19:26:30.741656+0000) 2019-12-09T19:27:06.614 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:27:06.607+0000 1694f700 -1 osd.2 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:48.075154+0000 front 2019-12-09T19:25:48.160974+0000 (oldest deadline 2019-12-09T19:27:02.139508+0000) 2019-12-09T19:27:06.624 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:27:06.620+0000 1694f700 -1 osd.7 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.460559+0000 front 2019-12-09T19:25:47.667553+0000 (oldest deadline 2019-12-09T19:26:33.429744+0000) 2019-12-09T19:27:06.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:06.817+0000 1694f700 -1 osd.5 320 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:25:49.416777+0000 front 2019-12-09T19:25:49.630618+0000 (oldest deadline 2019-12-09T19:26:31.998313+0000) 2019-12-09T19:27:27.351 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:06:19.201 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.354 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:06:19.204 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.392 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:06:19.217 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.408 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:06:19.233 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.419 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:06:19.269 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.427 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:06:19.252 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.432 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:06:19.256 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.476 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:06:19.300 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:27.485 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:06:19.310 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:27:36.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:36.929+0000 1694f700 -1 osd.6 322 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:118250 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821612096 gen 108] snapc 0=[] ondisk+write+known_if_redirected e321) 2019-12-09T19:27:41.247 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:41.216+0000 3578e700 -1 osd.5 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:41.242+0000 124b0700 -1 osd.5 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.438 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:41.387+0000 31f87700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.459 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:41.399+0000 11caf700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.521 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:41.365+0000 31f87700 -1 osd.5 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:27:41.449+0000 37f93700 -1 osd.5 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.656 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:41.610+0000 33f8b700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.675 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:27:41.647+0000 35f8f700 -1 osd.3 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:27:41.748 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:27:41.725+0000 34f8d700 -1 osd.6 324 build_incremental_map_msg missing incremental map 324 2019-12-09T19:28:04.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:04.570+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:04.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:04.782+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:04.849 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:04.845+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:04.987 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:04.982+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:05.588 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:05.584+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:05.835 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:05.828+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:05.846 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:05.841+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:05.994 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:05.987+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:06.582 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:06.577+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:06.644 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:06.640+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:06.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:06.843+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:06.878 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:06.872+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:06.989 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:06.984+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:07.166 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:07.162+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:07.590 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:07.586+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:07.691 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:07.685+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:07.837 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:07.832+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:07.903 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:07.896+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:08.048 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:08.041+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:08.129 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:08.124+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:08.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:08.611+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:08.708 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:08.701+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:08.884 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:08.880+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:08.898 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:08.893+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:09.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:09.008+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:09.152 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:09.145+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:09.587 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:09.584+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:09.684 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:09.681+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:09.881 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:09.877+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:09.888 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:09.883+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:10.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:10.029+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:10.205 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:10.200+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:10.567 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:10.562+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:10.732 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:10.727+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:10.839 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:10.835+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:10.858 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:10.853+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:11.023 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:11.018+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:11.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:11.192+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:11.221 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:11.218+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:11.578 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:11.574+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:11.717 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:11.713+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:11.822 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:11.816+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:11.890 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:11.887+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:12.014 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:12.006+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:12.227 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:12.220+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:12.248 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:12.243+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:12.625 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:12.622+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:12.730 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:12.726+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:12.870 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:12.864+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:12.924 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:12.919+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:12.982 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:12.977+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:13.209 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:13.204+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:13.236 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:13.231+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:13.615 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:13.612+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:13.725 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:13.721+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:13.893 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:13.888+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:13.972 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:13.968+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:14.023 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:14.019+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:14.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:14.194+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:14.230 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:14.224+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:14.660 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:14.656+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:14.700 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:14.696+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:14.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:14.887+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:14.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:14.976+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:14.990 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:14.985+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:15.206 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:15.200+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:15.225 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:15.221+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:15.711 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:15.707+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:15.712 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:15.708+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:15.914 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:15.907+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:16.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:16.004+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:16.023 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:16.019+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:16.183 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:16.176+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:16.200 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:16.196+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:16.708 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:16.704+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:16.718 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:16.713+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:16.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:16.921+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:16.998 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:16.993+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:17.043 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:17.040+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:17.180 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:17.174+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:17.246 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:17.242+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:17.699 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:17.695+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:17.746 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:17.741+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:17.944 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:17.936+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:17.956 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:17.950+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:17.996 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:17.993+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:18.185 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:18.179+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:18.266 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:18.260+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:18.733 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:18.728+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:18.770 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:18.766+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:18.906 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:18.900+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:18.922 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:18.917+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:18.962 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:18.957+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:19.192 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:19.186+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:19.288 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:19.283+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:19.291 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:19.287+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:19.744 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:19.740+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:19.816 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:19.812+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:19.879 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:19.874+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:19.915 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:19.909+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:19.919 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:19.915+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:20.242 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:20.236+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:20.315 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:20.309+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:20.316 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:20.311+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:20.731 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:20.727+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:20.779 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:20.775+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:20.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:20.868+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:20.952 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:20.947+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:20.959 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:20.953+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:21.287 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:21.281+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:21.356 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:21.349+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:21.357 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:21.352+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:21.756 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:21.751+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:21.809 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:21.804+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:21.849 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:21.843+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:21.949 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:21.944+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:21.969 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:21.963+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:22.249 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:22.244+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:22.316 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:22.309+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:22.318 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:22.312+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:22.772 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:22.768+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:22.798 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:22.795+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:22.817 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:22.813+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:22.945 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:22.940+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:22.971 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:22.967+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:23.247 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:23.242+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:23.281 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:23.274+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:23.282 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:23.277+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:23.758 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:23.754+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:23.829 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:23.825+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:23.835 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:23.830+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:23.930 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:23.925+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:23.991 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:23.986+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:24.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:24.257+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:24.282 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:24.276+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:24.283 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:24.279+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:24.787 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:24.782+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:24.818 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:24.814+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:24.840 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:24.836+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:24.948 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:24.943+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:24.995 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:24.990+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:25.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:25.234+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:25.241 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:25.235+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:25.244 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:25.239+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:25.751 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:25.745+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:25.789 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:25.783+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:25.870 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:25.863+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:25.920 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:25.914+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:26.028 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:26.023+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:26.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:26.288+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:26.296 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:26.290+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:26.297 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:26.292+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:26.785 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:26.781+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:26.799 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:26.793+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:26.837 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:26.833+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:26.918 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:26.913+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:27.021 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:27.015+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:27.267 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:27.262+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:27.314 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:27.309+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:27.315 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:27.310+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:27.812 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:27.806+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:27.821 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:27.815+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:27.836 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:27.830+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:27.889 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:27.884+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:28.060 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:28.054+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:28.310 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:28.305+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:28.348 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:28.342+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:28.349 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:28.344+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:28.826 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:28.821+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:28.854 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:28.849+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:28.863 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:28.858+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:28.932 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:28.926+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:29.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:29.012+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:29.318 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:29.311+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:29.319 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:29.314+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:29.356 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:29.351+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:29.821 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:29.816+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:29.827 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:29.822+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:29.851 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:29.846+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:29.903 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:29.897+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:29.999 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:29.994+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:30.292 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:30.284+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:30.293 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:30.287+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:30.328 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:30.323+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:30.789 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:30.784+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:30.843 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:30.839+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:30.870 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:30.865+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:30.890 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:30.885+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:31.053 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:31.047+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:31.327 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:31.322+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:31.331 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:31.327+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:31.368 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:31.362+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:31.759 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:31.754+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:31.801 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:31.796+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:31.853 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:31.847+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:31.915 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:31.910+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:32.052 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:32.046+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:32.311 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:32.305+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:32.312 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:32.308+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:32.421 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:32.416+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:32.750 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:32.745+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:32.779 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:32.775+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:32.828 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:32.824+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:32.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:32.918+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:33.102 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:33.093+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:33.320 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:33.313+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:33.320 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:33.316+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:33.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:33.392+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:33.781 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:33.771+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:33.794 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:33.789+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:33.857 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:33.851+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:33.957 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:33.951+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:34.070 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:34.063+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:34.353 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:34.347+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:34.353 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:34.349+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:34.417 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:34.412+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:34.744 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:34.738+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:34.791 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:34.781+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:34.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:34.891+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:34.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:34.993+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:35.048 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:35.041+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:35.327 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:35.319+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:35.328 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:35.321+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:35.467 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:35.462+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:35.753 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:35.748+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:35.757 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:35.753+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:35.907 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:35.901+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:36.012 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:36.007+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:36.040 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:36.035+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:36.341 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:36.336+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:36.348 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:36.341+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:36.522 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:36.516+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:36.751 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:36.746+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:36.798 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:36.793+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:36.891 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:36.885+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:36.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:36.968+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:37.085 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:37.077+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:37.319 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:37.314+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:37.322 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:37.318+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:37.485 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:28:37.478+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e326 v326) 2019-12-09T19:28:37.505 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:37.500+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:37.756 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:37.751+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:37.783 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:37.778+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:37.922 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:37.917+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:37.975 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:37.969+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:38.059 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:38.055+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:38.351 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:38.346+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:38.354 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:38.350+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:38.485 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:38.479+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:38.743 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:38.728+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:38.764 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:38.759+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:38.942 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:38.937+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:38.955 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:38.949+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:39.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:39.090+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:39.323 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:39.317+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:39.324 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:39.320+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:39.478 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:39.472+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:39.689 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:39.683+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:39.756 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:39.751+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:39.936 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:39.931+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:39.957 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:39.951+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:40.120 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:40.115+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:40.360 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:40.352+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:40.360 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:40.355+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:40.450 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:40.444+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:40.706 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:40.700+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:40.791 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:40.785+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:40.913 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:40.908+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:40.921 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:40.915+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:41.075 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:41.070+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:41.323 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:41.317+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:41.324 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:41.319+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:41.487 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:41.482+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:41.676 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:41.672+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:41.810 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:41.806+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:41.879 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:41.872+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:41.916 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:41.909+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:42.089 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:42.085+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:42.359 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:42.354+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:42.363 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:42.359+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:42.501 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:28:42.497+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e326 v326) 2019-12-09T19:28:42.523 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:42.517+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:42.673 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:42.669+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:42.788 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:42.783+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:42.881 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:42.869+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:42.961 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:42.955+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:43.070 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:43.064+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:43.357 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:43.350+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:43.358 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:43.353+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:43.567 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:43.561+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:43.673 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:43.668+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:43.810 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:43.805+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:43.885 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:43.879+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:43.950 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:43.944+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:44.055 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:44.048+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:44.319 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:44.313+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:44.320 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:44.315+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:44.586 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:44.581+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:44.680 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:44.675+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:44.804 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:44.799+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:44.855 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:44.849+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:44.980 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:44.975+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:45.012 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:45.007+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:45.336 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:45.331+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:45.337 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:45.333+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:45.584 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:45.579+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:45.652 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:45.647+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:45.778 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:45.774+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:45.881 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:45.870+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:45.976 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:45.962+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:46.021 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:46.016+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:46.359 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:46.355+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:46.360 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:46.357+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:46.577 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:46.572+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:46.666 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:46.661+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:46.733 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:46.729+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:46.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:46.862+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:46.951 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:46.946+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:47.019 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:47.015+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:47.385 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:47.379+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:47.386 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:47.382+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:47.528 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:28:47.519+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e326 v326) 2019-12-09T19:28:47.556 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:47.551+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:47.623 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:47.618+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:47.747 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:47.742+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:47.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:47.899+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:47.981 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:47.976+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:48.001 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:47.997+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:48.385 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:48.381+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:48.388 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:48.384+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:48.595 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:48.590+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:48.647 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:48.643+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:48.784 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:48.779+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:48.907 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:48.902+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:49.029 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:49.021+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:49.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:49.026+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:49.428 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:49.423+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:49.432 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:49.428+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:49.645 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:49.639+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:49.652 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:49.648+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:49.765 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:49.759+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:49.868 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:49.863+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:50.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:50.035+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:50.078 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:50.073+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:50.437 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:50.431+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:50.440 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:50.433+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:50.614 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:50.609+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:50.624 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:50.621+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:50.770 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:50.764+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:50.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:50.895+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:51.037 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:51.032+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:51.063 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:51.059+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:51.477 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:51.472+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:51.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:51.477+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:51.590 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:51.586+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:51.631 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:51.624+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:51.777 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:51.771+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:51.898 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:51.894+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:52.041 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:52.037+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:28:52.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:28:52.055+0000 1694f700 -1 osd.7 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.531003+0000 front 2019-12-09T19:27:21.298792+0000 (oldest deadline 2019-12-09T19:28:04.003965+0000) 2019-12-09T19:28:52.503 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:52.498+0000 1694f700 -1 osd.4 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:27:39.118256+0000 (oldest deadline 2019-12-09T19:28:19.118256+0000) 2019-12-09T19:28:52.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:28:52.503+0000 1694f700 -1 osd.4 326 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e324) 2019-12-09T19:28:52.547 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:28:52.544+0000 1694f700 -1 osd.0 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:21.721336+0000 front 2019-12-09T19:27:21.946749+0000 (oldest deadline 2019-12-09T19:28:05.623712+0000) 2019-12-09T19:28:52.678 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:28:52.672+0000 1694f700 -1 osd.5 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:22.278292+0000 front 2019-12-09T19:27:22.266807+0000 (oldest deadline 2019-12-09T19:28:06.948491+0000) 2019-12-09T19:28:52.680 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:28:52.600+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e326 v326) 2019-12-09T19:28:52.774 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:28:52.769+0000 1694f700 -1 osd.6 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:20.651148+0000 front 2019-12-09T19:27:20.621708+0000 (oldest deadline 2019-12-09T19:28:04.676856+0000) 2019-12-09T19:28:52.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:28:52.884+0000 1694f700 -1 osd.2 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.765759+0000 front 2019-12-09T19:27:18.743245+0000 (oldest deadline 2019-12-09T19:28:04.614577+0000) 2019-12-09T19:28:53.028 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:28:53.024+0000 1694f700 -1 osd.1 326 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:27:18.741103+0000 front 2019-12-09T19:27:18.233818+0000 (oldest deadline 2019-12-09T19:28:04.115902+0000) 2019-12-09T19:29:23.903 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:23.848+0000 34f8d700 -1 osd.3 331 build_incremental_map_msg missing incremental map 331 2019-12-09T19:29:23.991 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:23.869+0000 34f8d700 -1 osd.3 331 build_incremental_map_msg missing incremental map 331 2019-12-09T19:29:24.022 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:23.935+0000 35f8f700 -1 osd.3 331 build_incremental_map_msg missing incremental map 331 2019-12-09T19:29:24.459 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:24.337+0000 31f87700 -1 osd.3 331 build_incremental_map_msg missing incremental map 331 2019-12-09T19:29:24.467 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:24.340+0000 31f87700 -1 osd.3 331 build_incremental_map_msg missing incremental map 331 2019-12-09T19:29:27.204 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:29:27.197+0000 33f8b700 -1 osd.3 332 build_incremental_map_msg missing incremental map 331 2019-12-09T19:30:23.477 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:23.472+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:24.299 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:24.295+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:24.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:24.382+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:24.506 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:24.501+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:25.317 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:25.313+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:25.356 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:25.351+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:25.505 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:25.500+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:25.569 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:25.564+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:25.586 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:25.582+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:26.284 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:26.281+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:26.322 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:26.317+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:26.531 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:26.525+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:26.574 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:26.568+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:26.596 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:26.588+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:27.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:27.311+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:27.335 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:27.331+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:27.352 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:27.348+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:27.372 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:27.368+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:27.510 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:27.503+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:27.568 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:27.563+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:27.638 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:27.634+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:28.275 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:28.270+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:28.307 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:28.304+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:28.345 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:28.341+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:28.380 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:28.376+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:28.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:28.462+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:28.596 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:28.590+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:28.642 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:28.637+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:29.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:29.311+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:29.330 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:29.327+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:29.332 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:29.326+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:29.410 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:29.405+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:29.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:29.465+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:29.602 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:29.598+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:29.683 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:29.678+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:30.302 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:30.297+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:30.348 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:30.344+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:30.374 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:30.370+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:30.380 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:30.376+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:30.493 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:30.487+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:30.623 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:30.618+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:30.735 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:30.729+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:31.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:31.254+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:31.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:31.333+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:31.340 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:31.336+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:31.424 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:31.419+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:31.539 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:31.533+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:31.670 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:31.666+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:31.754 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:31.749+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:32.309 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:32.305+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:32.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:32.312+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:32.375 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:32.371+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:32.457 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:32.452+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:32.523 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:32.518+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:32.645 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:32.634+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:32.804 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:32.799+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:33.285 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:33.279+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:33.322 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:33.318+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:33.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:33.402+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:33.469 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:33.464+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:33.479 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:33.474+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:33.658 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:33.652+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:33.845 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:33.840+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:34.252 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:34.248+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:34.325 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:34.321+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:34.398 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:34.394+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:34.456 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:34.452+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:34.471 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:34.461+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:34.637 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:34.632+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:34.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:34.824+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:35.217 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:35.212+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:35.380 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:35.375+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:35.431 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:35.426+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:35.439 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:35.435+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:35.449 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:35.444+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:35.670 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:35.665+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:35.881 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:35.876+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:36.257 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:36.253+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:36.353 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:36.349+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:36.442 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:36.437+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:36.449 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:36.445+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:36.502 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:36.498+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:36.708 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:36.704+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:36.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:36.891+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:37.284 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:37.279+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:37.376 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:37.372+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:37.418 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:37.414+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:37.434 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:37.423+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:37.558 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:37.553+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:37.687 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:37.683+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:37.882 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:37.877+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:38.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:38.310+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:38.426 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:38.422+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:38.456 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:38.452+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:38.472 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:38.467+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:38.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:38.591+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:38.670 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:38.665+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:38.929 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:38.925+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:39.300 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:39.296+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:39.390 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:39.386+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:39.440 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:39.435+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:39.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:39.444+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:39.560 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:39.555+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:39.671 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:39.665+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:39.917 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:39.911+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:40.180 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:09:32.030 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.189 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:09:32.039 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.197 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:09:32.022 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.197 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:09:32.022 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.201 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:09:32.026 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.201 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:09:32.026 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.333 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:40.330+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:40.391 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:09:32.241 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.393 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:09:32.218 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.406 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:40.403+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:40.412 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:09:32.236 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:30:40.466 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:40.462+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:40.488 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:40.484+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:40.587 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:40.583+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:40.643 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:40.639+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:40.942 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:40.937+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:41.354 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:41.351+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:41.379 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:41.375+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:41.459 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:41.456+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:41.461 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:41.458+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:41.632 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:41.627+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:41.667 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:41.662+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:41.970 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:41.964+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:42.313 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:42.309+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:42.355 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:42.351+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:42.432 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:42.429+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:42.440 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:42.436+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:42.599 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:42.594+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:42.697 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:42.692+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:43.020 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:43.015+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:43.312 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:43.308+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:43.351 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:43.347+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:43.447 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:43.442+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:43.452 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:43.447+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:43.649 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:43.645+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:43.701 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:43.695+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:44.065 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:44.059+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:44.332 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:44.328+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:44.356 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:44.352+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:44.455 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:44.451+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:44.502 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:44.498+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:44.610 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:44.604+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:44.732 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:44.727+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:45.033 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:45.028+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:45.323 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:45.318+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:45.374 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:45.370+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:45.415 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:45.410+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:45.544 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:45.540+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:45.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:45.602+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:45.734 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:45.730+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:46.007 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:46.002+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:46.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:46.311+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:46.390 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:46.387+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:46.450 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:46.446+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:46.518 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:46.513+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:46.585 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:46.578+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:46.689 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:46.684+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:47.021 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:47.014+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:47.366 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:47.362+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:47.415 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:47.410+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:47.473 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:47.466+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:47.522 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:47.518+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:47.567 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:47.561+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:47.694 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:47.690+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:48.002 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:47.996+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:48.371 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:48.366+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:48.413 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:48.409+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:48.499 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:48.495+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:48.514 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:48.510+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:48.598 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:48.594+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:48.714 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:48.708+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:48.994 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:48.990+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:49.387 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:49.382+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:49.423 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:49.419+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:49.515 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:49.510+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:49.518 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:49.514+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:49.646 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:49.642+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:49.718 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:49.713+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:49.998 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:49.992+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:50.385 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:50.381+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:50.442 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:50.437+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:50.493 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:50.489+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:50.569 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:50.565+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:50.636 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:50.632+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:50.733 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:50.729+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:51.011 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:51.006+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:51.399 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:51.395+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:51.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:51.432+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:51.543 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:51.539+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:51.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:51.606+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:51.625 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:51.620+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:51.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:51.722+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:52.064 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:52.059+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:52.374 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:52.370+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:52.430 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:52.426+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:52.511 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:52.507+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:52.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:52.636+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:52.652 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:52.646+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:52.715 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:52.710+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:53.104 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:53.099+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:53.396 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:53.390+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:53.470 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:53.464+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:53.549 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:53.543+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:53.633 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:53.628+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:53.665 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:53.659+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:53.694 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:53.690+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:54.083 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:54.079+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:54.367 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:54.361+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:54.449 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:54.443+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:54.584 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:54.579+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:54.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:54.603+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:54.657 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:54.653+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:54.699 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:54.695+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:55.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:55.097+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:55.331 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:55.326+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:55.460 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:55.454+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:55.582 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:55.578+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:55.588 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:55.582+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:55.653 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:55.642+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:55.737 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:55.731+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:56.089 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:56.083+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:56.300 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:56.295+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:56.431 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:56.425+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:56.541 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:56.535+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:56.602 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:56.597+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:56.680 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:56.675+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:56.773 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:56.768+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:57.090 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:57.085+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:57.350 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:57.345+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:57.404 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:57.399+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:57.589 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:57.584+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:57.613 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:57.602+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:57.720 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:57.714+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:57.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:57.766+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:58.034 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:30:58.028+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e332 v332) 2019-12-09T19:30:58.124 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:58.120+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:58.379 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:58.375+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:58.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:58.374+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:58.573 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:58.567+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:58.635 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:58.630+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:58.767 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:58.761+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:30:58.770 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:58.765+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:59.167 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:30:59.162+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:30:59.352 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:30:59.347+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:30:59.397 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:30:59.392+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:30:59.593 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:30:59.588+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:30:59.607 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:30:59.602+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:30:59.782 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:30:59.776+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:30:59.805 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:30:59.800+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:31:00.138 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:31:00.134+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:31:00.354 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:31:00.349+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:31:00.361 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:00.356+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:31:00.570 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:31:00.566+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:31:00.593 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:00.589+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:31:00.743 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:31:00.739+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:31:00.843 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:31:00.839+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:31:01.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:31:01.094+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:31:01.371 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:31:01.367+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:31:01.386 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:01.381+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:31:01.592 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:31:01.587+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:31:01.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:01.622+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:31:01.759 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:31:01.754+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:31:01.811 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:31:01.806+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:31:02.138 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:31:02.133+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:31:02.357 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:31:02.352+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:31:02.378 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:02.371+0000 1694f700 -1 osd.3 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:39.950946+0000 front 2019-12-09T19:29:40.994973+0000 (oldest deadline 2019-12-09T19:30:23.389790+0000) 2019-12-09T19:31:02.584 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:02.579+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:31:02.611 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:31:02.606+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:31:02.807 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:31:02.803+0000 1694f700 -1 osd.6 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.732129+0000 front 2019-12-09T19:29:42.723556+0000 (oldest deadline 2019-12-09T19:30:24.975532+0000) 2019-12-09T19:31:02.854 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:31:02.849+0000 1694f700 -1 osd.2 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.568830+0000 front 2019-12-09T19:29:41.560709+0000 (oldest deadline 2019-12-09T19:30:26.836433+0000) 2019-12-09T19:31:03.112 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:31:03.097+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e332 v332) 2019-12-09T19:31:03.161 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:31:03.157+0000 1694f700 -1 osd.7 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:41.298371+0000 front 2019-12-09T19:29:42.548071+0000 (oldest deadline 2019-12-09T19:30:24.750546+0000) 2019-12-09T19:31:03.366 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:31:03.362+0000 1694f700 -1 osd.1 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:40.805548+0000 front 2019-12-09T19:29:40.981543+0000 (oldest deadline 2019-12-09T19:30:23.962033+0000) 2019-12-09T19:31:03.612 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:03.608+0000 1694f700 -1 osd.0 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.580137+0000 front 2019-12-09T19:29:41.686497+0000 (oldest deadline 2019-12-09T19:30:26.882509+0000) 2019-12-09T19:31:03.657 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:31:03.646+0000 1694f700 -1 osd.5 332 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:29:42.490412+0000 front 2019-12-09T19:29:42.487246+0000 (oldest deadline 2019-12-09T19:30:23.465846+0000) 2019-12-09T19:31:36.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:36.618+0000 30f85700 -1 osd.0 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:36.715 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:36.666+0000 31786700 -1 osd.0 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:36.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:36.739+0000 36790700 -1 osd.0 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:36.810 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:36.678+0000 3378a700 -1 osd.3 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:36.916 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:36.854+0000 32788700 -1 osd.3 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:37.012 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:31:36.922+0000 32788700 -1 osd.3 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:37.048 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:36.929+0000 36f91700 -1 osd.0 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:31:37.089 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:31:37.041+0000 38794700 -1 osd.0 337 build_incremental_map_msg missing incremental map 337 2019-12-09T19:32:01.832 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:01.826+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:02.059 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:02.056+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:02.886 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:02.877+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:03.027 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:03.023+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:03.191 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:03.188+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:03.940 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:03.933+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:04.042 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:04.036+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:04.151 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:04.146+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:04.909 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:04.902+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:04.994 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:04.991+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:05.125 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:05.120+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:05.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:05.149+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:05.155 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:05.150+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:05.755 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:05.752+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:05.891 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:05.882+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:05.957 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:05.953+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:06.097 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:06.089+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:06.122 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:06.117+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:06.129 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:06.125+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:06.736 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:06.732+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:06.877 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:06.870+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:06.957 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:06.953+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:07.116 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:07.111+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:07.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:07.143+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:07.156 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:07.151+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:07.195 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:07.191+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:07.720 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:07.715+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:07.921 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:07.916+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:07.946 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:07.940+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:08.075 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:08.070+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:08.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:08.137+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:08.159 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:08.154+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:08.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:08.155+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:08.745 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:08.738+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:08.892 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:08.887+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:08.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:08.897+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:09.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:09.093+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:09.152 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:09.146+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:09.157 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:09.151+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:09.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:09.195+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:09.746 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:09.740+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:09.867 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:09.862+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:09.877 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:09.872+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:10.070 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:10.065+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:10.136 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:10.129+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:10.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:10.164+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:10.177 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:10.170+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:10.713 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:10.708+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:10.894 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:10.890+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:10.929 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:10.923+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:11.085 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:11.080+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:11.139 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:11.134+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:11.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:11.156+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:11.170 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:11.165+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:11.699 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:11.693+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:11.882 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:11.876+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:11.902 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:11.897+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:12.132 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:12.127+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:12.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:12.132+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:12.143 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:12.138+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:12.201 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:12.195+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:12.726 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:12.721+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:12.873 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:12.866+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:12.944 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:12.939+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:13.112 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:13.106+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:13.158 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:13.153+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:13.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:13.171+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:13.214 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:13.210+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:13.710 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:13.704+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:13.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:13.823+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:13.952 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:13.947+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:14.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:14.143+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:14.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:14.148+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:14.217 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:14.212+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:14.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:14.221+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:14.702 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:14.697+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:14.815 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:14.810+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:15.006 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:15.001+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:15.148 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:15.143+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:15.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:15.149+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:15.230 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:15.225+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:15.234 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:15.230+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:15.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:15.237+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:15.734 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:15.728+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:15.844 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:15.837+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:16.000 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:15.994+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:16.104 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:16.101+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:16.148 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:16.143+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:16.209 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:16.204+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:16.240 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:16.234+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:16.241 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:16.235+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:16.769 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:16.764+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:16.812 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:16.806+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:16.956 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:16.950+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:17.113 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:17.104+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:17.154 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:17.149+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:17.173 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:17.168+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:17.244 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:17.239+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:17.247 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:17.243+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:17.776 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:17.771+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:17.865 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:17.859+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:18.014 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:18.004+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:18.162 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:18.157+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:18.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:18.179+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:18.210 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:18.205+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:18.224 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:18.219+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:18.227 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:18.223+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:18.811 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:18.806+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:18.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:18.871+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:19.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:19.055+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:19.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:19.133+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:19.171 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:19.164+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:19.202 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:19.197+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:19.203 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:19.198+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:19.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:19.219+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:19.806 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:19.801+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:19.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:19.916+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:20.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:20.039+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:20.142 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:20.137+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:20.177 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:20.171+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:20.187 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:20.183+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:20.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:20.194+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:20.200 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:20.196+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:20.818 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:20.813+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:20.965 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:20.958+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:21.057 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:21.052+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:21.174 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:21.170+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:21.185 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:21.177+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:21.219 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:21.214+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:21.222 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:21.218+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:21.223 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:21.220+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:21.825 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:21.820+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:21.953 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:21.948+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:22.091 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:22.085+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:22.165 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:22.159+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:22.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:22.200+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:22.238 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:22.233+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:22.242 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:22.238+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:22.252 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:22.247+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:22.825 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:22.819+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:22.983 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:22.979+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:23.110 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:23.105+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:23.153 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:23.148+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:23.187 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:23.180+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:23.224 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:23.217+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:23.225 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:23.220+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:23.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:23.235+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:23.863 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:23.858+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:23.938 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:23.932+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:24.110 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:24.104+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:24.132 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:24.127+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:24.153 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:24.147+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:24.202 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:24.196+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:24.203 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:24.198+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:24.271 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:24.266+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:24.854 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:24.847+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:24.985 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:24.980+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:25.076 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:25.071+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:25.128 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:25.122+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:25.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:25.154+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:25.196 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:25.190+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:25.197 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:25.192+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:25.311 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:25.306+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:25.825 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:25.820+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:26.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:26.018+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:26.072 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:26.066+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:26.088 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:26.082+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:26.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:26.132+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:26.174 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:26.167+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:26.175 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:26.170+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:26.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:26.289+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:26.806 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:26.800+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:26.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:26.984+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:27.057 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:27.052+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:27.109 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:27.105+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:27.164 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:27.158+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:27.199 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:27.194+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:27.202 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:27.198+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:27.279 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:27.274+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:27.860 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:27.853+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:28.001 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:27.996+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:28.057 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:28.052+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:28.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:28.096+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:28.182 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:28.176+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:28.245 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:28.239+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:28.246 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:28.242+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:28.282 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:28.277+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:28.850 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:28.844+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:29.024 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:29.018+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:29.062 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:29.054+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:29.086 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:29.081+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:29.178 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:29.173+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:29.219 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:29.214+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:29.221 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:29.218+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:29.275 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:29.269+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:29.816 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:29.811+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:30.054 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:30.049+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:30.069 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:30.063+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:30.101 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:30.096+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:30.201 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:30.195+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:30.277 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:30.269+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:30.278 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:30.271+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:30.292 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:30.286+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:30.792 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:30.785+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:31.026 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:31.020+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:31.044 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:31.039+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:31.112 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:31.107+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:31.247 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:31.241+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:31.271 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:31.265+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:31.272 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:31.267+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:31.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:31.308+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:31.844 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:31.839+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:31.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:31.987+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:32.016 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:32.009+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:32.103 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:32.097+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:32.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:32.206+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:32.302 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:32.297+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:32.305 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:32.300+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:32.308 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:32.304+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:32.855 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:32.851+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:32.995 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:32.990+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:33.017 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:33.012+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:33.115 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:33.109+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:33.219 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:33.215+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:33.303 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:33.298+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:33.335 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:33.330+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:33.336 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:33.332+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:33.889 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:33.885+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:33.987 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:33.976+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:34.003 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:33.998+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:34.156 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:34.148+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:34.249 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:34.242+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:34.354 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:34.349+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:34.378 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:34.372+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:34.379 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:34.374+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:34.873 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:34.869+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:34.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:34.966+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:35.010 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:35.005+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:35.157 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:35.151+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:35.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:35.223+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:35.362 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:35.357+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:35.388 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:35.382+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:35.389 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:35.384+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:35.890 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:35.885+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:35.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:35.971+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:36.031 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:36.025+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:36.181 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:36.175+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:36.244 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:36.240+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:36.370 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:36.365+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:36.432 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:36.426+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:36.433 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:36.428+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:36.873 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:36.868+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:36.993 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:36.988+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:37.030 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:37.025+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:37.198 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:37.193+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:37.211 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:37.206+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:37.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:37.372+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:37.462 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:37.456+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:37.468 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:37.458+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:37.870 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:37.865+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:37.968 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:37.957+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:38.024 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:38.019+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:38.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:38.161+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:38.184 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:38.178+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:38.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:38.343+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:38.448 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:38.442+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:38.450 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:38.445+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:38.489 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:32:38.479+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e339 v339) 2019-12-09T19:32:38.868 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:38.864+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:39.014 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:39.009+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:39.021 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:39.016+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:39.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:39.158+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:39.186 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:39.181+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:39.349 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:39.344+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:39.457 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:39.452+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:39.462 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:39.456+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:39.922 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:39.916+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:39.996 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:39.992+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:40.073 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:40.067+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:40.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:40.173+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:40.211 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:40.206+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:40.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:40.388+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:40.475 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:40.470+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:40.480 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:40.474+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:40.940 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:40.936+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:40.961 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:40.956+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:41.100 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:41.096+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:41.166 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:41.163+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:41.182 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:41.176+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:41.445 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:41.438+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:41.446 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:41.439+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:41.446 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:41.441+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:41.908 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:41.904+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:41.987 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:41.983+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:42.155 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:42.150+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:42.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:42.159+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:42.211 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:42.206+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:42.432 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:42.426+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:42.435 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:42.429+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:42.441 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:42.432+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:42.942 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:42.938+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:42.997 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:42.991+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:43.128 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:43.123+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:43.201 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:43.196+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:43.202 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:43.196+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:43.433 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:43.427+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:43.434 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:43.429+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:43.468 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:43.462+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:43.515 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:32:43.501+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e339 v339) 2019-12-09T19:32:43.897 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:43.892+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:43.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:43.969+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:44.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:44.144+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:44.166 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:44.160+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:44.207 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:44.202+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:44.440 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:44.435+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:44.465 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:44.460+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:44.466 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:44.462+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:44.945 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:44.941+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:44.966 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:44.962+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:45.108 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:45.104+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:45.152 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:45.146+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:45.189 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:45.183+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:45.449 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:45.444+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:45.453 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:45.449+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:45.474 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:45.468+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:45.921 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:45.916+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:45.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:45.934+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:46.087 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:46.081+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:46.149 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:46.137+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:46.167 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:46.163+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:46.429 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:46.423+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:46.493 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:46.488+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:46.500 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:46.492+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:46.944 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:46.940+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:46.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:46.968+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:47.058 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:47.053+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:47.107 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:47.102+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:47.129 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:47.126+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:47.408 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:47.402+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:47.534 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:47.527+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:47.535 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:47.530+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:47.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:32:47.925+0000 1694f700 -1 osd.2 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.436985+0000 front 2019-12-09T19:31:19.526189+0000 (oldest deadline 2019-12-09T19:32:01.108824+0000) 2019-12-09T19:32:47.948 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:47.943+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:48.026 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:32:48.021+0000 1694f700 -1 osd.6 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.227860+0000 front 2019-12-09T19:31:20.182120+0000 (oldest deadline 2019-12-09T19:32:04.867275+0000) 2019-12-09T19:32:48.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:32:48.095+0000 1694f700 -1 osd.7 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:15.800452+0000 front 2019-12-09T19:31:15.798552+0000 (oldest deadline 2019-12-09T19:32:01.008817+0000) 2019-12-09T19:32:48.162 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:48.158+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:32:48.380 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:32:48.371+0000 1694f700 -1 osd.5 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.572759+0000 front 2019-12-09T19:31:20.534958+0000 (oldest deadline 2019-12-09T19:32:04.613199+0000) 2019-12-09T19:32:48.538 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:48.532+0000 1694f700 -1 osd.4 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:31:34.663850+0000 (oldest deadline 2019-12-09T19:32:14.663850+0000) 2019-12-09T19:32:48.545 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:32:48.534+0000 1694f700 -1 osd.4 339 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e337) 2019-12-09T19:32:48.607 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:32:48.599+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e339 v339) 2019-12-09T19:32:48.947 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:32:48.943+0000 1694f700 -1 osd.1 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:20.781520+0000 front 2019-12-09T19:31:20.877166+0000 (oldest deadline 2019-12-09T19:32:05.459004+0000) 2019-12-09T19:32:49.118 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:32:49.115+0000 1694f700 -1 osd.0 339 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:31:19.900976+0000 front 2019-12-09T19:31:19.849595+0000 (oldest deadline 2019-12-09T19:32:02.727274+0000) 2019-12-09T19:33:17.875 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:17.866+0000 32f89700 -1 osd.3 339 build_incremental_map_msg missing incremental map 337 2019-12-09T19:33:23.978 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:12:15.828 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:23.997 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:12:15.847 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.012 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:12:15.837 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.031 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:12:15.856 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.045 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:12:15.870 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.047 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:12:15.872 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.184 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:12:16.034 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.308 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.166+0000 30f85700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:24.329 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.168+0000 30f85700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:24.336 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:12:16.161 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.342 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.196+0000 36f91700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:24.344 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:12:16.170 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:33:24.373 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.255+0000 34f8d700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:24.406 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.277+0000 34f8d700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:24.888 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:24.757+0000 31786700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:33:25.793 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:33:25.775+0000 37f93700 -1 osd.3 345 build_incremental_map_msg missing incremental map 345 2019-12-09T19:34:24.562 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:24.558+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:25.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:25.026+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:25.541 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:25.537+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:26.068 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:26.065+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:26.128 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:26.123+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:26.534 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:26.530+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:26.823 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:26.817+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:27.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:27.078+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:27.116 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:27.113+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:27.522 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:27.518+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:27.829 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:27.824+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:28.006 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:28.002+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:28.074 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:28.070+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:28.088 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:28.084+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:28.117 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:28.113+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:28.142 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:28.137+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:28.521 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:28.517+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:28.806 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:28.802+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:29.046 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:29.042+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:29.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:29.078+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:29.102 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:29.095+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:29.122 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:29.117+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:29.189 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:29.185+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:29.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:29.543+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:29.828 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:29.824+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:30.076 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:30.066+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:30.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:30.075+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:30.101 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:30.097+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:30.141 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:30.125+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:30.234 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:30.229+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:30.569 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:30.565+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:30.850 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:30.845+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:31.067 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:31.063+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:31.081 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:31.076+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:31.092 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:31.087+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:31.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:31.175+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:31.258 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:31.254+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:31.531 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:31.527+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:31.808 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:31.803+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:32.053 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:32.048+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:32.059 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:32.055+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:32.094 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:32.089+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:32.133 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:32.129+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:32.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:32.221+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:32.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:32.545+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:32.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:32.781+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:33.043 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:33.038+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:33.063 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:33.059+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:33.091 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:33.087+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:33.123 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:33.119+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:33.187 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:33.181+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:33.538 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:33.534+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:33.781 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:33.776+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:34.039 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:34.035+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:34.060 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:34.056+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:34.082 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:34.077+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:34.131 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:34.125+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:34.158 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:34.153+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:34.300 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:13:26.150 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.306 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:13:26.156 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.312 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:13:26.137 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.315 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:13:26.140 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.317 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:13:26.142 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.322 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:13:26.146 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.399 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:13:26.223 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.399 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:13:26.224 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.441 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:13:26.288 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:34:34.549 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:34.545+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:34.833 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:34.829+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:35.034 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:35.029+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:35.056 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:35.052+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:35.089 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:35.085+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:35.093 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:35.087+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:35.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:35.182+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:35.594 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:35.590+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:35.821 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:35.817+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:35.996 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:35.991+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:36.074 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:36.068+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:36.081 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:36.074+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:36.081 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:36.070+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:36.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:36.221+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:36.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:36.607+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:36.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:36.826+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:37.029 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:37.024+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:37.096 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:37.090+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:37.105 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:37.097+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:37.109 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:37.104+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:37.256 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:37.251+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:37.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:37.607+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:37.812 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:37.807+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:38.059 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:38.054+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:38.118 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:38.106+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:38.130 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:38.126+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:38.132 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:38.127+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:38.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:38.282+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:38.656 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:38.651+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:38.779 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:38.774+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:39.018 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:39.012+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:39.091 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:39.086+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:39.145 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:39.140+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:39.167 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:39.160+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:39.269 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:39.264+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:39.680 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:39.674+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:39.790 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:39.785+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:40.040 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:40.034+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:40.084 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:40.078+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:40.104 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:40.100+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:40.180 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:40.175+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:40.245 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:40.241+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:40.674 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:40.668+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:40.766 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:40.762+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:41.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:41.039+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:41.068 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:41.062+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:41.118 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:41.112+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:41.145 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:41.140+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:41.267 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:41.261+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:41.703 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:41.698+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:41.765 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:41.757+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:42.085 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:42.079+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:42.091 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:42.087+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:42.101 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:42.096+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:42.134 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:42.128+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:42.275 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:42.270+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:42.728 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:42.723+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:42.734 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:42.730+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:43.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:43.047+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:43.113 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:43.108+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:43.117 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:43.113+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:43.177 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:43.172+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:43.280 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:43.275+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:43.752 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:43.744+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:43.770 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:43.765+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:44.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:44.003+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:44.077 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:44.070+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:44.102 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:44.096+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:44.145 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:44.139+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:44.307 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:44.302+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:44.782 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:44.777+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:44.805 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:44.800+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:44.987 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:44.982+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:45.112 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:45.106+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:45.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:45.126+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:45.145 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:45.139+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:45.274 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:45.269+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:45.760 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:45.755+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:45.833 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:45.826+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:46.015 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:46.011+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:46.124 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:46.116+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:46.146 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:46.141+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:46.170 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:46.163+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:46.233 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:46.228+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:46.775 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:46.770+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:46.793 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:46.782+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:46.973 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:46.968+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:47.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:47.133+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:47.162 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:47.158+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:47.175 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:47.171+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:47.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:47.201+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:47.742 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:47.737+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:47.848 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:47.827+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:47.936 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:47.931+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:48.122 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:48.117+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:48.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:48.124+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:48.132 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:48.127+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:48.231 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:48.225+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:48.794 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:48.788+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:48.803 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:48.798+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:48.982 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:48.977+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:49.100 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:49.095+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:49.148 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:49.142+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:49.159 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:49.155+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:49.279 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:49.275+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:49.799 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:49.794+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:49.837 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:49.832+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:50.010 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:50.005+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:50.079 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:50.073+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:50.125 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:50.120+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:50.173 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:50.167+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:50.260 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:50.255+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:50.812 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:50.807+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:50.821 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:50.814+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:51.045 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:51.031+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:51.123 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:51.118+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:51.133 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:51.128+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:51.151 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:51.146+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:51.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:51.261+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:51.803 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:51.797+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:51.859 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:51.853+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:52.036 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:52.031+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:52.090 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:52.085+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:52.138 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:52.132+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:52.184 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:52.179+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:52.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:52.281+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:52.775 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:52.770+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:52.899 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:52.893+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:53.022 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:53.019+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:53.066 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:53.061+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:53.118 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:53.114+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:53.213 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:53.203+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:53.330 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:53.325+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:53.822 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:53.818+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:53.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:53.882+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:54.055 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:54.051+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:54.072 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:54.068+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:54.144 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:54.139+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:54.269 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:54.264+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:54.296 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:54.292+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:54.868 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:54.863+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:54.940 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:54.936+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:55.094 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:55.088+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:55.108 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:55.103+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:55.198 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:55.193+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:55.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:55.297+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:55.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:55.309+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:55.923 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:55.918+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:55.949 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:55.940+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:56.067 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:56.062+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:56.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:56.077+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:56.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:56.156+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:56.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:56.259+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:56.279 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:56.275+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:56.937 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:56.931+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:56.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:56.968+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:57.118 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:57.112+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:57.121 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:57.116+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:57.204 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:57.198+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:57.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:57.308+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:57.316 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:57.310+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:57.924 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:57.919+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:57.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:57.984+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:58.100 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:58.073+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:58.100 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:58.072+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:58.245 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:58.239+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:58.338 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:58.333+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:58.363 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:58.358+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:58.901 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:58.897+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:58.969 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:58.964+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:34:59.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:34:59.107+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:34:59.113 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:34:59.109+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:34:59.236 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:34:59.232+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e345 v345) 2019-12-09T19:34:59.256 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:34:59.251+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:34:59.331 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:34:59.327+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:34:59.410 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:34:59.405+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:34:59.932 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:34:59.927+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:34:59.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:34:59.969+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:35:00.120 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:35:00.115+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:35:00.135 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:35:00.131+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:35:00.226 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:35:00.223+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:35:00.355 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:00.352+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:35:00.379 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:35:00.375+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:35:00.902 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:35:00.897+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:35:01.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:35:01.014+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:35:01.145 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:35:01.142+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:35:01.147 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:35:01.141+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:35:01.261 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:35:01.256+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:35:01.390 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:01.386+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:35:01.433 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:35:01.428+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:35:01.898 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:35:01.893+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:35:02.051 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:35:02.047+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:35:02.130 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:35:02.123+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:35:02.196 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:35:02.191+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:35:02.217 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:35:02.214+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:35:02.367 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:02.363+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:35:02.390 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:35:02.385+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:35:02.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:35:02.885+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:35:03.052 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:35:03.047+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:35:03.145 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:35:03.141+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:35:03.188 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:35:03.184+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:35:03.224 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:35:03.218+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:35:03.379 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:03.374+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:35:03.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:35:03.384+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:35:03.872 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:35:03.867+0000 1694f700 -1 osd.7 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.834459+0000 front 2019-12-09T19:33:40.602720+0000 (oldest deadline 2019-12-09T19:34:25.880293+0000) 2019-12-09T19:35:04.073 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:35:04.061+0000 1694f700 -1 osd.2 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.005630+0000 front 2019-12-09T19:33:44.046245+0000 (oldest deadline 2019-12-09T19:34:24.254680+0000) 2019-12-09T19:35:04.150 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:35:04.144+0000 1694f700 -1 osd.3 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.496553+0000 front 2019-12-09T19:33:43.163653+0000 (oldest deadline 2019-12-09T19:34:25.969083+0000) 2019-12-09T19:35:04.211 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:35:04.207+0000 1694f700 -1 osd.0 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:42.370829+0000 front 2019-12-09T19:33:42.368965+0000 (oldest deadline 2019-12-09T19:34:27.334814+0000) 2019-12-09T19:35:04.225 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:35:04.221+0000 1694f700 -1 osd.5 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:44.004354+0000 front 2019-12-09T19:33:43.495876+0000 (oldest deadline 2019-12-09T19:34:27.836929+0000) 2019-12-09T19:35:04.302 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:35:04.296+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e345 v345) 2019-12-09T19:35:04.359 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:04.349+0000 1694f700 -1 osd.1 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:40.348298+0000 front 2019-12-09T19:33:40.343241+0000 (oldest deadline 2019-12-09T19:34:24.314016+0000) 2019-12-09T19:35:04.391 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:35:04.386+0000 1694f700 -1 osd.6 345 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:33:43.081137+0000 front 2019-12-09T19:33:42.950345+0000 (oldest deadline 2019-12-09T19:34:27.648400+0000) 2019-12-09T19:35:37.772 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:37.725+0000 36b90700 -1 osd.1 349 build_incremental_map_msg missing incremental map 349 2019-12-09T19:35:37.895 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:37.811+0000 38b94700 -1 osd.1 349 build_incremental_map_msg missing incremental map 349 2019-12-09T19:35:37.945 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:35:37.899+0000 38b94700 -1 osd.1 349 build_incremental_map_msg missing incremental map 349 2019-12-09T19:35:38.677 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:35:38.634+0000 31f87700 -1 osd.4 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:35:39.801 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:35:39.774+0000 32788700 -1 osd.4 350 build_incremental_map_msg missing incremental map 350 2019-12-09T19:35:49.656 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:14:41.506 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.696 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:14:41.546 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.701 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:14:41.526 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.705 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:14:41.530 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.727 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:14:41.552 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.735 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:14:41.560 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.796 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:14:41.646 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.836 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:14:41.661 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:35:49.866 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:14:41.691 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:36:02.362 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:02.358+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:02.823 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:02.818+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:03.050 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:03.043+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:03.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:03.336+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:03.831 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:03.827+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:04.040 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:04.035+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:04.334 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:04.329+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:04.805 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:04.800+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:05.001 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:04.997+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:05.285 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:05.280+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:05.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:05.341+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:05.395 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:05.390+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:05.851 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:05.847+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:05.879 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:05.875+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:05.985 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:05.980+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:06.330 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:06.325+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:06.352 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:06.347+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:06.395 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:06.389+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:06.850 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:06.845+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:06.894 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:06.889+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:06.945 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:06.939+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:07.353 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:07.349+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:07.356 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:07.350+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:07.403 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:07.399+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:07.850 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:07.844+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:07.897 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:07.891+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:07.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:07.924+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:08.313 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:08.307+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:08.368 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:08.364+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:08.374 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:08.369+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:08.491 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:08.487+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:08.868 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:08.864+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:08.883 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:08.878+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:08.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:08.910+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:09.345 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:09.339+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:09.353 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:09.348+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:09.357 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:09.351+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:09.487 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:09.483+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:09.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:09.903+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:09.910 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:09.906+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:09.934 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:09.925+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:10.331 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:10.326+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:10.333 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:10.327+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:10.383 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:10.377+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:10.535 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:10.531+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:10.892 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:10.887+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:10.934 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:10.929+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:10.958 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:10.953+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:11.294 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:11.289+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:11.306 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:11.301+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:11.432 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:11.428+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:11.533 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:11.527+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:11.878 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:11.873+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:11.934 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:11.930+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:11.965 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:11.961+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:12.279 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:12.274+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:12.347 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:12.342+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:12.390 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:12.385+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:12.571 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:12.566+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:12.863 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:12.858+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:12.944 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:12.939+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:12.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:12.959+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:13.289 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:13.283+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:13.355 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:13.350+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:13.390 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:13.384+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:13.546 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:13.541+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:13.826 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:13.820+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:13.924 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:13.919+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:13.944 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:13.939+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:14.264 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:14.258+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:14.404 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:14.399+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:14.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:14.403+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:14.522 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:14.517+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:14.858 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:14.854+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:14.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:14.917+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:14.994 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:14.988+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:15.248 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:15.242+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:15.394 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:15.379+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:15.394 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:15.388+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:15.482 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:15.475+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:15.483 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:15.478+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:15.831 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:15.827+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:15.889 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:15.885+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:16.029 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:16.024+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:16.293 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:16.287+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:16.381 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:16.377+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:16.388 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:16.383+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:16.437 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:16.431+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:16.438 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:16.433+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:16.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:16.842+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:16.851 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:16.846+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:16.991 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:16.985+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:17.260 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:17.255+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:17.372 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:17.369+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:17.386 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:17.380+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:17.424 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:17.418+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:17.425 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:17.420+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:17.870 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:17.866+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:17.898 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:17.894+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:18.028 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:18.023+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:18.262 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:18.257+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:18.379 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:18.376+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:18.417 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:18.412+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:18.473 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:18.469+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:18.476 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:18.473+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:18.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:18.862+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:18.897 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:18.892+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:18.982 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:18.978+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:19.223 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:19.217+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:19.418 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:19.414+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:19.443 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:19.431+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:19.445 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:19.439+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:19.450 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:19.444+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:19.889 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:19.884+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:19.950 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:19.946+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:19.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:19.968+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:20.209 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:20.203+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:20.418 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:20.414+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:20.442 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:20.437+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:20.465 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:20.460+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:20.468 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:20.465+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:20.844 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:20.840+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:20.934 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:20.928+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:20.965 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:20.960+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:21.233 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:21.228+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:21.413 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:21.409+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:21.419 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:21.414+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:21.462 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:21.456+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:21.464 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:21.459+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:21.893 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:21.890+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:21.905 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:21.899+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:21.975 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:21.970+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:22.264 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:22.258+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:22.375 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:22.371+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:22.456 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:22.449+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:22.484 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:22.479+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:22.488 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:22.484+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:22.857 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:22.853+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:22.955 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:22.949+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:23.012 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:23.008+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:23.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:23.235+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:23.388 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:23.383+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:23.457 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:23.451+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:23.458 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:23.454+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:23.497 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:23.492+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:23.818 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:23.815+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:23.945 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:23.940+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:24.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:23.999+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:24.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:24.222+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:24.348 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:24.343+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:24.441 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:24.436+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:24.445 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:24.441+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:24.473 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:24.466+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:24.780 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:24.776+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:24.939 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:24.935+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:24.995 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:24.991+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:25.193 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:25.188+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:25.357 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:25.353+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:25.417 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:25.410+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:25.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:25.413+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:25.470 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:25.464+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:25.813 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:25.809+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:25.917 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:25.911+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:26.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:25.998+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:26.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:26.223+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:26.319 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:26.315+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:26.406 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:26.400+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:26.410 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:26.406+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:26.444 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:26.439+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:26.768 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:26.763+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:26.881 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:26.876+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:27.010 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:27.006+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:27.245 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:27.238+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:27.316 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:27.312+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:27.417 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:27.409+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:27.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:27.412+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:27.487 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:27.482+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:27.738 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:27.735+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:27.847 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:27.841+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:27.974 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:27.970+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:28.237 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:28.232+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:28.326 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:28.322+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:28.425 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:28.419+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:28.426 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:28.421+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:28.473 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:28.466+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:28.706 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:28.701+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:28.828 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:28.823+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:29.002 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:28.997+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:29.249 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:29.243+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:29.323 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:29.319+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:29.433 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:29.427+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:29.451 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:29.446+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:29.455 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:29.451+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:29.667 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:29.661+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:29.880 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:29.875+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:30.032 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:30.027+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:30.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:30.289+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:30.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:30.297+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:30.455 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:30.449+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:30.456 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:30.451+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:30.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:30.455+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:30.699 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:30.695+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:30.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:30.860+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:31.035 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:31.031+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:31.256 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:31.253+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:31.317 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:31.312+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:31.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:31.415+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:31.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:31.499+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:31.506 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:31.502+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:31.673 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:31.668+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:31.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:31.860+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:32.032 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:32.028+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:32.306 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:32.302+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:32.362 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:32.354+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:32.440 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:32.434+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:32.494 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:32.488+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:32.495 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:32.490+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:32.688 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:32.684+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:32.846 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:32.841+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:33.044 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:33.040+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:33.352 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:33.348+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:33.408 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:33.403+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:33.414 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:33.408+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:33.515 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:33.510+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:33.516 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:33.512+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:33.722 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:33.717+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:33.851 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:33.845+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:34.046 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:34.042+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:34.390 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:34.387+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:34.424 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:34.419+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:34.428 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:34.423+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:34.502 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:34.498+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:34.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:34.501+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:34.687 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:34.684+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:34.889 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:34.885+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:35.046 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:35.041+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:35.356 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:35.349+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:35.435 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:35.430+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:35.482 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:35.478+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:35.538 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:35.533+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:35.542 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:35.538+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:35.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:35.667+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:35.847 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:35.842+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:36.013 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:36.008+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:36.323 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:36.319+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:36.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:36.392+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:36.460 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:36.452+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:36.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:36.505+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:36.511 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:36.508+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:36.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:36.710+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:36.891 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:36.886+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:37.059 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:37.056+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:37.296 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:37.294+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:37.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:37.389+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:37.483 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:37.475+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:37.522 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:37.519+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:37.529 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:37.523+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:37.731 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:37.727+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:37.917 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:37.913+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:38.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:38.032+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:38.263 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:38.260+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:38.415 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:38.410+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:38.435 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:38.430+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:38.538 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:38.532+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:38.539 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:38.534+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:38.723 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:38.719+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:38.898 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:38.892+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:39.013 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:39.009+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:39.309 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:39.304+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:39.435 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:39.429+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:39.438 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:39.433+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:39.559 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:39.553+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:39.560 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:39.556+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:39.696 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:39.689+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:39.744 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:36:39.737+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e351 v351) 2019-12-09T19:36:39.864 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:39.859+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:40.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:40.048+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:40.267 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:40.261+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:40.424 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:40.419+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:40.454 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:40.448+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:40.526 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:40.521+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:40.529 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:40.525+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:40.696 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:40.693+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:40.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:40.901+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:41.048 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:41.044+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:41.302 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:41.299+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:41.382 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:41.378+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:41.452 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:41.447+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:41.523 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:41.516+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:41.524 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:41.519+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:41.696 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:41.692+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:41.922 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:41.917+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:42.030 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:42.026+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:42.348 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:42.345+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:42.414 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:42.409+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:42.506 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:42.499+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:42.519 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:42.513+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:42.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:42.515+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:42.737 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:42.732+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:42.925 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:42.920+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:43.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:43.000+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:43.350 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:43.346+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:43.396 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:43.391+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:43.464 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:43.458+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:43.525 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:43.519+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:43.525 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:43.521+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:43.693 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:43.688+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:43.894 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:43.888+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:44.032 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:44.029+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:44.309 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:44.304+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:44.402 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:44.396+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:44.443 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:44.436+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:44.494 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:44.488+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:44.495 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:44.491+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:44.686 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:44.679+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:44.771 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:36:44.758+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e351 v351) 2019-12-09T19:36:44.895 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:44.891+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:44.991 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:44.988+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:45.323 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:45.319+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:45.403 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:45.398+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:45.486 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:45.480+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:45.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:45.502+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:45.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:45.504+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:45.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:45.710+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:45.882 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:45.877+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:45.994 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:45.989+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:46.361 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:46.357+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:46.375 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:46.370+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:46.506 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:46.500+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:46.514 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:46.509+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:46.515 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:46.511+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:46.747 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:46.742+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:46.940 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:46.934+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:47.001 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:46.997+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:47.335 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:47.331+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:47.404 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:47.399+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:47.470 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:47.465+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:47.493 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:47.488+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:47.497 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:47.493+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:47.705 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:47.698+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:47.945 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:47.939+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:47.996 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:47.992+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:48.338 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:48.335+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:48.432 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:48.427+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:48.492 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:48.487+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:48.496 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:48.492+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:48.497 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:48.492+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:48.713 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:48.708+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:48.956 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:48.944+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:48.972 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:48.968+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:49.306 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:49.301+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:49.434 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:36:49.428+0000 1694f700 -1 osd.5 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:20.834996+0000 front 2019-12-09T19:35:20.169295+0000 (oldest deadline 2019-12-09T19:36:05.148964+0000) 2019-12-09T19:36:49.539 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:49.534+0000 1694f700 -1 osd.4 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:35:35.207852+0000 (oldest deadline 2019-12-09T19:36:15.207852+0000) 2019-12-09T19:36:49.541 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:49.538+0000 1694f700 -1 osd.4 351 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:49.542 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:36:49.537+0000 1694f700 -1 osd.7 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.424004+0000 front 2019-12-09T19:35:19.424883+0000 (oldest deadline 2019-12-09T19:36:05.238455+0000) 2019-12-09T19:36:49.682 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:36:49.679+0000 1694f700 -1 osd.2 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:19.721020+0000 front 2019-12-09T19:35:19.497799+0000 (oldest deadline 2019-12-09T19:36:02.298809+0000) 2019-12-09T19:36:49.846 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:36:49.836+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e351 v351) 2019-12-09T19:36:49.931 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:36:49.927+0000 1694f700 -1 osd.1 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:21.903524+0000 front 2019-12-09T19:35:21.924664+0000 (oldest deadline 2019-12-09T19:36:05.296442+0000) 2019-12-09T19:36:49.955 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:36:49.951+0000 1694f700 -1 osd.6 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.980521+0000 front 2019-12-09T19:35:18.982937+0000 (oldest deadline 2019-12-09T19:36:02.162453+0000) 2019-12-09T19:36:50.326 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:36:50.322+0000 1694f700 -1 osd.0 351 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:35:18.812898+0000 front 2019-12-09T19:35:18.983213+0000 (oldest deadline 2019-12-09T19:36:02.246147+0000) 2019-12-09T19:36:50.665 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:50.585+0000 1694f700 -1 osd.4 352 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e349) 2019-12-09T19:36:50.697 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:36:50.662+0000 36f91700 -1 osd.4 352 build_incremental_map_msg missing incremental map 350 2019-12-09T19:38:22.188 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:22.181+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:22.560 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:22.554+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:22.890 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:22.884+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:23.111 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:23.107+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:23.144 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:23.140+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:23.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:23.157+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:23.578 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:23.573+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:23.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:23.910+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:24.109 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:24.103+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:24.117 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:24.113+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:24.136 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:24.131+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:24.621 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:24.615+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:24.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:24.896+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:25.049 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:25.045+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:25.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:25.112+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:25.122 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:25.113+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:25.128 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:25.120+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:25.665 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:25.660+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:25.870 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:25.865+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:25.917 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:25.912+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:26.035 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:26.030+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:26.090 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:26.085+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:26.097 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:26.091+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:26.156 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:26.150+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:26.643 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:26.639+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:26.896 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:26.891+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:26.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:26.951+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:27.006 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:27.002+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:27.131 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:27.126+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:27.139 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:27.135+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:27.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:27.160+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:27.623 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:27.619+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:27.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:27.909+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:27.964 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:27.958+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:28.031 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:28.025+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:28.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:28.151+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:28.160 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:28.156+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:28.189 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:28.184+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:28.664 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:28.659+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:28.929 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:28.924+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:28.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:28.967+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:29.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:29.004+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:29.156 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:29.151+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:29.199 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:29.194+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:29.207 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:29.202+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:29.687 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:29.682+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:29.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:29.915+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:29.935 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:29.928+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:30.031 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:30.027+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:30.162 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:30.157+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:30.179 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:30.173+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:30.185 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:30.179+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:30.709 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:30.703+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:30.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:30.943+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:30.969 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:30.964+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:31.054 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:31.049+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:31.153 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:31.147+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:31.164 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:31.159+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:31.175 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:31.170+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:31.704 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:31.699+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:31.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:31.980+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:32.008 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:32.003+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:32.101 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:32.096+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:32.136 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:32.131+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:32.171 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:32.166+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:32.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:32.182+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:32.667 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:32.662+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:32.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:32.971+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:33.033 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:33.028+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:33.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:33.130+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:33.152 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:33.147+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:33.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:33.149+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:33.219 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:33.213+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:33.625 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:33.620+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:33.946 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:33.941+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:34.057 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:34.052+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:34.110 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:34.105+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:34.144 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:34.139+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:34.180 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:34.176+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:34.214 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:34.208+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:34.612 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:34.607+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:34.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:34.909+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:35.031 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:35.026+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:35.086 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:35.081+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:35.117 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:35.111+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:35.143 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:35.138+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:35.251 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:35.246+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:35.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:35.612+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:35.882 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:35.878+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:36.046 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:36.038+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:36.058 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:36.053+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:36.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:36.134+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:36.194 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:36.189+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:36.245 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:36.240+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:36.582 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:36.577+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:36.878 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:36.872+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:37.030 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:37.025+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:37.065 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:37.060+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:37.126 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:37.121+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:37.202 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:37.196+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:37.221 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:37.217+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:37.612 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:37.607+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:37.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:37.919+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:38.011 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:38.006+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:38.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:38.109+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:38.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:38.157+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:38.192 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:38.186+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:38.240 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:38.222+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:38.635 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:38.630+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:38.899 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:38.894+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:39.013 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:39.009+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:39.139 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:39.134+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:39.194 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:39.189+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:39.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:39.194+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:39.198 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:39.194+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:39.233 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:17:31.083 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.246 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:17:31.096 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.250 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:17:31.075 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.253 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:17:31.078 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.258 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:17:31.083 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.258 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:17:31.083 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.415 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:17:31.265 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.421 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:17:31.246 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.425 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:17:31.250 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:38:39.599 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:39.595+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:39.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:39.916+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:40.059 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:40.054+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:40.119 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:40.114+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:40.154 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:40.150+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:40.180 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:40.175+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:40.229 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:40.225+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:40.552 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:40.548+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:40.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:40.882+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:41.068 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:41.064+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:41.108 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:41.103+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:41.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:41.141+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:41.204 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:41.197+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:41.210 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:41.206+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:41.542 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:41.538+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:41.917 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:41.912+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:42.113 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:42.108+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:42.122 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:42.113+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:42.185 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:42.179+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:42.190 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:42.183+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:42.219 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:42.214+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:42.508 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:42.502+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:42.935 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:42.930+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:43.130 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:43.125+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:43.147 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:43.142+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:43.150 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:43.147+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:43.217 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:43.212+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:43.253 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:43.249+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:43.512 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:43.508+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:43.956 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:43.952+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:44.099 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:44.095+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:44.121 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:44.117+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:44.164 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:44.160+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:44.256 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:44.250+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:44.264 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:44.260+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:44.468 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:44.464+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:44.962 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:44.957+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:45.127 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:45.124+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:45.154 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:45.150+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:45.159 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:45.156+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:45.213 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:45.209+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:45.250 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:45.246+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:45.485 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:45.482+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:45.985 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:45.980+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:46.143 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:46.137+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:46.173 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:46.165+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:46.174 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:46.167+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:46.176 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:46.171+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:46.215 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:46.211+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:46.508 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:46.503+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:47.031 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:47.027+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:47.149 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:47.144+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:47.156 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:47.152+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:47.197 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:47.192+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:47.219 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:47.211+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:47.222 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:47.217+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:47.475 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:47.471+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:48.039 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:48.034+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:48.148 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:48.143+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:48.195 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:48.189+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:48.212 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:48.208+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:48.245 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:48.240+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:48.273 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:48.269+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:48.464 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:48.460+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:49.057 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:49.053+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:49.108 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:49.104+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:49.195 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:49.191+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:49.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:49.236+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:49.248 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:49.245+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:49.317 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:49.313+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:49.481 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:49.475+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:50.034 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:50.029+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:50.112 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:50.108+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:50.230 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:50.225+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:50.235 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:50.231+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:50.243 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:50.238+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:50.305 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:50.301+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:50.528 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:50.524+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:50.995 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:50.992+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:51.072 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:51.068+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:51.221 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:51.215+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:51.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:51.234+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:51.251 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:51.247+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:51.294 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:51.289+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:51.504 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:51.499+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:51.952 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:51.948+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:52.036 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:52.030+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:52.213 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:52.208+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:52.239 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:52.234+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:52.262 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:52.258+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:52.325 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:52.320+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:52.557 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:52.552+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:52.998 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:52.992+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:53.037 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:53.032+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:53.189 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:53.184+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:53.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:53.193+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:53.240 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:53.235+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:53.300 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:53.296+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:53.546 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:53.541+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:53.981 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:53.977+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:54.028 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:54.023+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:54.149 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:54.144+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:54.154 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:54.149+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:54.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:54.231+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:54.306 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:54.302+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:54.531 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:54.527+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:54.965 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:54.962+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:55.025 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:55.020+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:55.113 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:55.108+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:55.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:55.133+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:55.196 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:55.192+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:55.317 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:55.312+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:55.390 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:38:55.386+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 41sec e357 v357) 2019-12-09T19:38:55.489 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:55.485+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:55.991 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:55.987+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:56.015 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:56.011+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:56.106 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:56.101+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:56.114 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:56.109+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:56.209 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:56.205+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:56.294 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:56.290+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:56.465 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:56.459+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:56.948 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:56.944+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:57.051 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:57.047+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:57.097 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:57.094+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:57.156 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:57.152+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:57.259 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:57.256+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:57.296 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:57.292+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:57.446 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:57.443+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:57.935 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:57.929+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:58.021 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:58.017+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:58.152 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:58.147+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:58.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:58.181+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:58.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:58.268+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:58.348 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:58.342+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:58.441 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:58.438+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:58.955 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:58.951+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:38:59.072 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:38:59.068+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:38:59.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:38:59.133+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:38:59.140 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:38:59.136+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:38:59.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:38:59.296+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:38:59.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:38:59.310+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:38:59.405 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:38:59.401+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:38:59.942 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:38:59.937+0000 1694f700 -1 osd.7 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.083216+0000 front 2019-12-09T19:37:39.826509+0000 (oldest deadline 2019-12-09T19:38:22.121862+0000) 2019-12-09T19:39:00.103 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:39:00.099+0000 1694f700 -1 osd.2 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:41.718961+0000 front 2019-12-09T19:37:41.534502+0000 (oldest deadline 2019-12-09T19:38:22.020371+0000) 2019-12-09T19:39:00.132 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:00.128+0000 1694f700 -1 osd.5 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:40.616303+0000 front 2019-12-09T19:37:40.635954+0000 (oldest deadline 2019-12-09T19:38:24.691933+0000) 2019-12-09T19:39:00.175 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:00.170+0000 1694f700 -1 osd.6 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.794843+0000 front 2019-12-09T19:37:38.625294+0000 (oldest deadline 2019-12-09T19:38:22.677348+0000) 2019-12-09T19:39:00.289 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:39:00.285+0000 1694f700 -1 osd.0 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:39.545421+0000 front 2019-12-09T19:37:40.223747+0000 (oldest deadline 2019-12-09T19:38:25.389007+0000) 2019-12-09T19:39:00.339 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:39:00.332+0000 1694f700 -1 osd.3 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:38.082259+0000 front 2019-12-09T19:37:39.784304+0000 (oldest deadline 2019-12-09T19:38:23.071682+0000) 2019-12-09T19:39:00.395 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:39:00.392+0000 1694f700 -1 osd.1 357 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:37:42.049162+0000 front 2019-12-09T19:37:36.379953+0000 (oldest deadline 2019-12-09T19:38:21.920595+0000) 2019-12-09T19:39:00.441 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:39:00.438+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 41sec e357 v357) 2019-12-09T19:39:32.053 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:31.913+0000 32788700 -1 osd.6 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.168 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:31.954+0000 35f8f700 -1 osd.6 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.229 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:32.220+0000 35f8f700 -1 osd.5 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:32.285+0000 3578e700 -1 osd.5 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:32.149+0000 33f8b700 -1 osd.6 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.404 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:32.327+0000 33f8b700 -1 osd.6 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.418 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:32.391+0000 38f95700 -1 osd.6 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:32.450 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:32.299+0000 37f93700 -1 osd.5 361 build_incremental_map_msg missing incremental map 361 2019-12-09T19:39:57.681 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:39:57.676+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:39:57.845 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:57.839+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:39:58.660 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:39:58.653+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:39:58.690 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:39:58.687+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:39:58.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:58.798+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:39:59.300 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:39:59.294+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:39:59.670 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:39:59.666+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:39:59.674 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:39:59.669+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:39:59.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:39:59.825+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:39:59.904 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:39:59.898+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:00.356 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:00.351+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:00.633 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:00.628+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:00.662 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:00.658+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:00.799 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:00.795+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:00.868 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:00.863+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:01.362 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:01.357+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:01.607 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:01.604+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:01.684 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:01.681+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:01.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:01.769+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:01.902 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:01.898+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:02.354 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:02.349+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:02.579 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:02.574+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:02.659 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:02.655+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:02.706 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:02.702+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:02.817 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:02.812+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:02.930 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:02.925+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:02.994 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:02.990+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:03.333 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:03.328+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:03.579 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:03.575+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:03.678 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:03.674+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:03.698 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:03.693+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:03.782 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:03.777+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:03.955 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:03.952+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:03.964 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:03.959+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:04.300 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:04.295+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:04.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:04.553+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:04.670 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:04.666+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:04.735 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:04.732+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:04.841 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:04.829+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:04.946 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:04.941+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:04.950 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:04.946+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:05.280 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:05.275+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:05.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:05.547+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:05.705 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:05.698+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:05.765 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:05.758+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:05.816 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:05.812+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:05.910 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:05.903+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:05.933 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:05.929+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:06.275 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:06.270+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:06.551 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:06.547+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:06.725 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:06.721+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:06.798 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:06.794+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:06.862 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:06.856+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:06.940 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:06.936+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:06.949 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:06.943+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:07.271 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:07.266+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:07.575 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:07.570+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:07.697 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:07.692+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:07.824 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:07.820+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:07.876 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:07.870+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:07.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:07.910+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:07.953 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:07.946+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:08.245 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:08.239+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:08.568 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:08.564+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:08.683 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:08.680+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:08.867 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:08.863+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:08.871 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:08.865+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:08.912 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:08.906+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:08.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:08.928+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:09.233 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:09.228+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:09.563 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:09.560+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:09.670 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:09.666+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:09.824 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:09.820+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:09.904 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:09.900+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:09.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:09.910+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:09.920 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:09.916+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:10.246 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:10.242+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:10.597 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:10.593+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:10.709 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:10.704+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:10.711 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:10.707+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:10.816 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:10.812+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:10.880 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:10.876+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:10.885 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:10.881+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:10.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:10.899+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:11.220 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:11.215+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:11.627 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:11.623+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:11.705 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:11.699+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:11.706 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:11.702+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:11.769 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:11.765+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:11.839 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:11.835+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:11.928 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:11.923+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:11.931 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:11.925+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:12.242 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:12.237+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:12.602 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:12.598+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:12.739 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:12.735+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:12.751 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:12.746+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:12.752 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:12.748+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:12.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:12.833+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:12.900 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:12.895+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:12.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:12.918+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:13.218 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:13.213+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:13.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:13.562+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:13.751 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:13.734+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:13.752 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:13.739+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:13.787 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:13.784+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:13.808 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:13.803+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:13.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:13.897+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:13.921 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:13.916+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:14.205 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:14.191+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:14.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:14.565+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:14.747 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:14.739+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:14.748 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:14.741+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:14.761 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:14.755+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:14.846 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:14.841+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:14.870 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:14.866+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:14.970 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:14.964+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:15.226 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:15.221+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:15.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:15.615+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:15.792 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:15.788+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:15.795 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:15.791+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:15.807 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:15.802+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:15.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:15.833+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:15.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:15.896+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:16.000 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:15.995+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:16.228 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:16.223+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:16.591 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:16.587+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:16.758 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:16.752+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:16.759 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:16.755+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:16.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:16.762+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:16.837 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:16.831+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:16.898 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:16.894+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:16.998 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:16.992+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:17.265 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:17.261+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:17.598 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:17.594+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:17.764 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:17.758+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:17.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:17.760+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:17.791 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:17.785+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:17.814 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:17.809+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:17.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:17.859+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:17.990 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:17.983+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:18.316 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:18.311+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:18.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:18.614+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:18.779 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:18.774+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:18.784 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:18.780+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:18.794 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:18.790+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:18.819 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:18.813+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:18.867 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:18.863+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:19.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:19.003+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:19.367 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:19.362+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:19.593 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:19.589+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:19.782 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:19.772+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:19.782 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:19.775+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:19.800 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:19.794+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:19.826 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:19.822+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:19.859 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:19.855+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:20.000 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:19.995+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:20.388 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:20.382+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:20.552 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:20.548+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:20.798 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:20.792+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:20.799 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:20.794+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:20.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:20.806+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:20.829 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:20.825+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:20.845 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:20.841+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:21.032 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:21.026+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:21.386 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:21.382+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:21.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:21.569+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:21.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:21.759+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:21.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:21.761+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:21.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:21.789+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:21.859 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:21.856+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:21.881 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:21.878+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:22.067 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:22.062+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:22.434 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:22.422+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:22.599 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:22.594+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:22.761 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:22.755+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:22.765 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:22.760+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:22.784 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:22.779+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:22.899 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:22.894+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:22.911 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:22.906+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:23.080 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:23.075+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:23.398 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:23.392+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:23.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:23.565+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:23.806 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:23.801+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:23.806 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:23.803+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:23.836 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:23.830+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:23.901 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:23.897+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:23.912 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:23.909+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:24.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:24.129+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:24.427 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:24.421+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:24.591 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:24.587+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:24.798 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:24.792+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:24.800 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:24.795+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:24.860 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:24.854+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:24.933 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:24.928+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:24.938 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:24.934+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:25.183 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:25.178+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:25.451 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:25.446+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:25.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:25.613+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:25.811 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:25.806+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:25.815 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:25.811+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:25.912 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:25.907+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:25.963 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:25.959+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:25.977 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:25.974+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:26.184 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:26.178+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:26.469 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:26.459+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:26.631 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:26.627+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:26.809 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:26.804+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:26.810 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:26.806+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:26.933 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:26.923+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:26.964 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:26.959+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:26.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:26.985+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:27.212 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:27.207+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:27.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:27.435+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:27.592 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:27.588+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:27.806 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:27.800+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:27.807 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:27.803+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:27.960 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:27.955+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:28.002 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:27.998+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:28.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:28.014+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:28.182 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:28.177+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:28.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:28.408+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:28.630 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:28.625+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:28.800 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:28.795+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:28.803 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:28.799+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:28.932 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:28.927+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:29.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:29.017+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:29.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:29.028+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:29.169 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:29.163+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:29.386 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:29.381+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:29.665 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:29.660+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:29.826 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:29.816+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:29.827 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:29.818+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:29.932 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:29.926+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:29.994 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:29.990+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:30.000 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:29.996+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:30.219 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:30.214+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:30.345 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:30.340+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:30.713 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:30.709+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:30.746 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:40:30.740+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e363 v363) 2019-12-09T19:40:30.804 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:30.800+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:30.808 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:30.804+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:30.960 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:30.954+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:30.981 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:30.977+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:31.007 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:31.004+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:31.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:31.238+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:31.332 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:31.327+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:31.697 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:31.693+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:31.811 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:31.805+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:31.811 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:31.807+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:32.005 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:32.002+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:32.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:32.003+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:32.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:32.028+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:32.294 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:32.289+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:32.356 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:32.351+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:32.657 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:32.652+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:32.827 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:32.823+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:32.832 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:32.827+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:32.965 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:32.959+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:33.023 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:33.017+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:33.030 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:33.025+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:33.301 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:33.296+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:33.377 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:33.372+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:33.691 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:33.688+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:33.873 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:33.868+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:33.877 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:33.873+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:33.978 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:33.972+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:34.037 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:34.021+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:34.037 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:34.030+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:34.273 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:34.268+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:34.333 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:34.328+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:34.675 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:34.670+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:34.886 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:34.881+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:34.887 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:34.883+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:34.937 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:34.931+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:35.002 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:34.998+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:35.070 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:35.066+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:35.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:35.228+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:35.320 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:35.315+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:35.722 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:35.718+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:35.765 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:40:35.759+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e363 v363) 2019-12-09T19:40:35.866 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:35.862+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:35.869 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:35.865+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:35.911 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:35.903+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:35.976 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:35.973+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:36.121 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:36.115+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:36.199 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:36.194+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:36.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:36.300+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:36.725 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:36.720+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:36.901 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:36.896+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:36.903 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:36.898+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:36.952 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:36.946+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:36.978 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:36.972+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:37.171 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:37.168+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:37.247 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:37.243+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:37.293 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:37.289+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:37.765 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:37.761+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:37.854 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:37.850+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:37.853+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:37.997 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:37.992+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:37.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:37.993+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:38.222 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:38.218+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:38.230 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:38.225+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:38.308 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:38.303+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:38.767 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:38.763+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:38.829 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:38.823+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:38.831 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:38.826+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:38.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:38.952+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:38.993 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:38.988+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:39.187 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:39.181+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:39.231 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:39.228+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:39.270 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:39.265+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:39.808 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:39.804+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:39.844 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:39.838+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:39.845 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:39.840+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:39.923 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:39.920+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:40.019 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:40.014+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:40.226 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:40.221+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:40.236 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:40.231+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:40.285 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:40.279+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:40.786 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:40.781+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:40.788 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:40:40.780+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e363 v363) 2019-12-09T19:40:40.888 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:40.882+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:40.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:40.885+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:40.912 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:40.908+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:40.989 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:40.983+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:41.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:41.220+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:41.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:41.259+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:41.317 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:41.314+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:41.789 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:41.785+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:41.931 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:41.925+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:41.931 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:41.927+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:41.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:41.943+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:42.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:42.003+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:42.268 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:42.264+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:42.295 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:42.290+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:42.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:42.350+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:42.797 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:42.791+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:42.892 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:42.883+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:42.892 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:42.888+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:42.918 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:42.913+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:43.008 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:43.003+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:43.285 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:43.280+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:43.307 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:43.303+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:43.343 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:43.338+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:43.753 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:43.749+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:43.894 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:43.889+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:43.920 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:43.915+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:43.921 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:43.916+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:43.996 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:43.991+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:44.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:44.306+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:44.328 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:44.323+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:44.387 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:44.382+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:44.741 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:44.734+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:44.862 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:44.857+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:44.885 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:44.880+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:44.889 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:44.885+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:45.048 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:45.042+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:45.351 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:45.347+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:40:45.360 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:40:45.355+0000 1694f700 -1 osd.2 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:17.437233+0000 front 2019-12-09T19:39:17.825826+0000 (oldest deadline 2019-12-09T19:40:01.995002+0000) 2019-12-09T19:40:45.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:40:45.356+0000 1694f700 -1 osd.6 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.821986+0000 front 2019-12-09T19:39:14.410268+0000 (oldest deadline 2019-12-09T19:39:59.024674+0000) 2019-12-09T19:40:45.753 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:40:45.748+0000 1694f700 -1 osd.1 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:15.149739+0000 front 2019-12-09T19:39:15.751452+0000 (oldest deadline 2019-12-09T19:39:57.851142+0000) 2019-12-09T19:40:45.869 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:40:45.863+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e363 v363) 2019-12-09T19:40:45.881 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:40:45.878+0000 1694f700 -1 osd.0 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:12.430692+0000 front 2019-12-09T19:39:17.835626+0000 (oldest deadline 2019-12-09T19:39:57.591632+0000) 2019-12-09T19:40:45.950 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:45.944+0000 1694f700 -1 osd.4 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:39:30.232047+0000 (oldest deadline 2019-12-09T19:40:10.232047+0000) 2019-12-09T19:40:45.954 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:40:45.947+0000 1694f700 -1 osd.4 363 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:117350 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e361) 2019-12-09T19:40:46.057 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:40:46.053+0000 1694f700 -1 osd.5 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:16.941817+0000 front 2019-12-09T19:39:16.934363+0000 (oldest deadline 2019-12-09T19:39:56.917553+0000) 2019-12-09T19:40:46.306 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:40:46.301+0000 1694f700 -1 osd.7 363 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:39:13.823539+0000 front 2019-12-09T19:39:13.871037+0000 (oldest deadline 2019-12-09T19:39:59.054307+0000) 2019-12-09T19:42:17.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:17.152+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:18.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:18.182+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:18.789 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:18.784+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:19.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:19.219+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:19.547 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:19.542+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:19.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:19.783+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:20.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:20.261+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:20.521 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:20.516+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:20.568 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:20.563+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:20.770 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:20.765+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:20.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:20.783+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:21.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:21.167+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:21.232 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:21.226+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:21.243 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:21.238+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:21.491 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:21.486+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:21.541 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:21.536+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:21.728 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:21.722+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:21.786 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:21.781+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:22.210 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:22.204+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:22.232 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:22.226+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:22.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:22.236+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:22.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:22.505+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:22.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:22.542+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:22.741 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:22.735+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:22.752 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:22.748+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:23.198 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:23.193+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:23.212 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:23.206+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:23.279 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:23.274+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:23.524 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:23.515+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:23.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:23.568+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:23.722 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:23.717+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:23.778 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:23.773+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:24.236 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:24.231+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:24.258 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:24.252+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:24.293 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:24.285+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:24.530 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:24.525+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:24.563 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:24.558+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:24.763 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:24.758+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:24.769 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:24.765+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:25.216 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:25.212+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:25.294 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:25.289+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:25.345 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:25.334+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:25.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:25.550+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:25.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:25.574+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:25.759 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:25.755+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:25.803 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:25.798+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:26.188 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:26.183+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:26.277 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:26.273+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:26.339 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:26.336+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:26.591 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:26.587+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:26.601 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:26.597+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:26.774 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:26.770+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:26.790 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:26.785+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:27.188 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:27.184+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:27.306 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:27.301+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:27.376 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:27.372+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:27.582 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:27.579+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:27.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:27.620+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:27.780 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:27.775+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:27.798 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:27.792+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:28.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:28.211+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:28.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:28.282+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:28.376 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:28.372+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:28.583 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:28.579+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:28.607 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:28.603+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:28.750 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:28.745+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:28.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:28.773+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:29.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:29.259+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:29.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:29.260+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:29.383 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:29.380+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:29.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:29.551+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:29.655 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:29.652+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:29.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:29.725+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:29.769 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:29.765+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:30.259 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:30.255+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:30.308 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:30.303+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:30.351 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:30.347+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:30.566 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:30.562+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:30.697 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:30.689+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:30.697 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:30.686+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:30.741 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:30.736+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:31.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:31.261+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:31.271 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:31.266+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:31.366 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:31.362+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:31.584 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:31.579+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:31.672 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:31.668+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:31.710 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:31.706+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:31.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:31.722+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:32.258 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:32.253+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:32.283 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:32.279+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:32.393 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:32.388+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:32.637 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:32.629+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:32.669 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:32.660+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:32.757 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:32.752+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:32.765 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:32.758+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:33.293 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:33.288+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:33.320 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:33.316+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:33.397 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:33.394+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:33.620 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:33.616+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:33.665 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:33.660+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:33.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:33.710+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:33.798 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:33.792+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:34.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:34.270+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:34.353 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:34.348+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:34.443 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:34.438+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:34.629 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:34.624+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:34.667 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:34.661+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:34.713 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:34.709+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:34.807 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:34.801+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:35.276 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:35.271+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:35.371 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:35.368+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:35.441 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:35.438+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:35.645 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:35.640+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:35.657 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:35.652+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:35.732 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:35.726+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:35.860 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:35.854+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:36.269 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:36.264+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:36.377 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:36.371+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:36.412 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:36.408+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:36.611 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:36.607+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:36.673 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:36.666+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:36.789 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:36.778+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:36.825 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:36.817+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:37.262 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:37.258+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:37.395 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:37.390+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:37.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:37.398+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:37.621 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:37.617+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:37.664 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:37.660+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:37.766 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:37.761+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:37.785 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:37.779+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:38.252 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:38.246+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:38.401 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:38.397+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:38.459 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:38.452+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:38.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:38.586+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:38.653 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:38.646+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:38.755 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:38.751+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:38.784 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:38.777+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:39.207 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:39.202+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:39.407 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:39.403+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:39.417 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:39.412+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:39.560 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:39.556+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:39.628 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:39.625+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:39.751 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:39.744+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:39.800 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:39.796+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:40.205 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:40.199+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:40.434 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:40.429+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:40.438 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:40.433+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:40.602 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:40.598+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:40.663 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:40.659+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:40.758 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:40.754+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:40.784 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:40.779+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:41.208 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:41.202+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:41.405 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:41.395+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:41.443 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:41.436+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:41.600 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:41.593+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:41.609 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:21:33.454 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.611 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:21:33.461 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.620 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:21:33.444 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.627 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:41.622+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:41.627 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:21:33.452 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.628 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:21:33.452 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.632 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:21:33.457 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.729 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:41.726+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:41.761 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:41.756+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:41.863 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:21:33.709 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.873 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:21:33.696 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:41.993 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:21:33.819 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:42:42.188 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:42.184+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:42.417 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:42.411+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:42.420 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:42.416+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:42.585 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:42.579+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:42.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:42.601+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:42.763 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:42.759+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:42.764 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:42.759+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:43.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:43.160+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:43.392 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:43.387+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:43.410 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:43.404+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:43.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:43.602+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:43.626 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:43.618+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:43.728 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:43.723+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:43.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:43.735+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:44.149 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:44.144+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:44.418 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:44.409+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:44.435 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:44.431+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:44.606 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:44.601+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:44.637 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:44.632+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:44.717 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:44.713+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:44.735 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:44.731+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:45.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:45.140+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:45.412 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:45.408+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:45.487 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:45.483+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:45.622 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:45.618+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:45.684 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:45.680+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:45.713 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:45.701+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:45.739 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:45.729+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:46.104 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:46.096+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:46.423 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:46.419+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:46.474 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:46.469+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:46.612 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:46.607+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:46.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:46.699+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:46.722 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:46.716+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:46.725 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:46.720+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:47.062 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:47.056+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:47.444 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:47.440+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:47.471 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:47.467+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:47.662 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:47.657+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:47.736 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:47.730+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:47.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:47.740+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:47.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:47.760+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:48.023 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:48.018+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:48.409 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:48.401+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:48.481 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:48.475+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:48.699 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:48.695+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:48.741 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:48.736+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:48.769 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:48.765+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:48.817 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:48.812+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:49.021 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:49.016+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:49.430 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:49.427+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:49.515 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:49.512+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:49.691 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:49.686+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:49.778 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:49.774+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:49.791 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:49.786+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:49.805 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:49.800+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:49.999 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:49.994+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:50.386 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:50.382+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:50.514 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:50.510+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:50.738 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:50.733+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:50.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:50.736+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:50.758 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:50.753+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:50.843 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:50.839+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:51.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:51.029+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:51.311 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:42:51.306+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e369 v369) 2019-12-09T19:42:51.436 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:51.432+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:51.497 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:51.493+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:51.730 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:51.724+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:51.773 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:51.770+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:51.777 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:51.774+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:51.828 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:51.823+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:52.015 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:52.010+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:52.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:52.397+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:52.472 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:52.464+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:52.683 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:52.679+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:52.740 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:52.736+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:52.783 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:52.779+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:52.790 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:52.785+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:53.030 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:53.025+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:53.411 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:53.402+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:53.523 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:53.519+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:53.715 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:53.710+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:53.779 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:53.766+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:53.786 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:53.781+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:53.803 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:53.798+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:54.018 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:54.014+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:54.421 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:54.416+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:54.547 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:54.542+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:54.744 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:54.738+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:54.758 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:54.753+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:54.770 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:54.765+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:54.817 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:54.813+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:54.995 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:54.990+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:55.457 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:55.454+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:42:55.568 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:42:55.565+0000 1694f700 -1 osd.0 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:38.837219+0000 front 2019-12-09T19:41:37.169456+0000 (oldest deadline 2019-12-09T19:42:20.790033+0000) 2019-12-09T19:42:55.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:42:55.748+0000 1694f700 -1 osd.7 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:37.262395+0000 front 2019-12-09T19:41:37.435919+0000 (oldest deadline 2019-12-09T19:42:20.027444+0000) 2019-12-09T19:42:55.757 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:42:55.745+0000 1694f700 -1 osd.3 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.470231+0000 front 2019-12-09T19:41:36.432667+0000 (oldest deadline 2019-12-09T19:42:20.504185+0000) 2019-12-09T19:42:55.763 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:42:55.760+0000 1694f700 -1 osd.2 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:35.627095+0000 front 2019-12-09T19:41:35.539534+0000 (oldest deadline 2019-12-09T19:42:18.689111+0000) 2019-12-09T19:42:55.786 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:42:55.776+0000 1694f700 -1 osd.5 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.468514+0000 front 2019-12-09T19:41:36.831654+0000 (oldest deadline 2019-12-09T19:42:18.419676+0000) 2019-12-09T19:42:56.000 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:42:55.996+0000 1694f700 -1 osd.6 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.500988+0000 front 2019-12-09T19:41:36.494104+0000 (oldest deadline 2019-12-09T19:42:16.474069+0000) 2019-12-09T19:42:56.387 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:42:56.382+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 40sec e369 v369) 2019-12-09T19:42:56.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:42:56.467+0000 1694f700 -1 osd.1 369 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:41:36.340650+0000 front 2019-12-09T19:41:36.469112+0000 (oldest deadline 2019-12-09T19:42:20.920457+0000) 2019-12-09T19:43:32.151 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.078+0000 36790700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:32.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.148+0000 34f8d700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:32.403 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.291+0000 31786700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:32.588 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.364+0000 32f89700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:32.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.484+0000 124b0700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:32.665 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:32.573+0000 114ae700 -1 osd.0 373 build_incremental_map_msg missing incremental map 373 2019-12-09T19:43:54.458 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:54.454+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:43:55.467 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:55.462+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:43:55.912 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:43:55.904+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:43:56.210 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:43:56.206+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:43:56.489 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:56.485+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:43:56.955 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:43:56.950+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:43:57.255 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:43:57.251+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:43:57.276 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:43:57.273+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:43:57.543 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:57.537+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:43:57.932 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:43:57.927+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:43:58.243 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:43:58.240+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:43:58.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:43:58.304+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:43:58.435 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:43:58.430+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:43:58.551 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:58.547+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:43:58.984 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:43:58.979+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:43:59.292 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:43:59.287+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:43:59.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:43:59.344+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:43:59.397 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:43:59.392+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:43:59.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:43:59.572+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:00.027 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:00.022+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:00.041 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:00.035+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:00.320 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:00.317+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:00.344 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:00.341+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:00.371 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:00.366+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:00.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:00.603+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:00.991 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:00.986+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:01.076 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:01.071+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:01.368 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:01.363+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:01.381 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:01.375+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:01.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:01.380+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:01.649 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:01.643+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:01.953 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:01.948+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:02.059 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:02.054+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:02.352 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:02.347+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:02.385 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:02.381+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:02.403 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:02.392+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:02.656 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:02.651+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:03.005 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:03.001+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:03.070 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:03.065+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:03.357 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:03.351+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:03.363 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:03.360+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:03.396 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:03.391+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:03.661 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:03.658+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:04.010 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:04.004+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:04.096 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:04.090+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:04.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:04.351+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:04.382 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:04.379+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:04.388 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:04.384+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:04.489 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:04.486+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:04.637 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:04.633+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:05.046 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:05.040+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:05.114 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:05.108+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:05.340 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:05.334+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:05.355 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:05.351+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:05.434 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:05.430+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:05.502 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:05.499+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:05.609 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:05.606+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:06.053 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:06.049+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:06.160 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:06.152+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:06.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:06.306+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:06.335 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:06.331+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:06.460 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:06.456+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:06.531 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:06.528+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:06.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:06.593+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:07.083 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:07.078+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:07.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:07.202+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:07.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:07.293+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:07.388 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:07.382+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:07.504 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:07.500+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:07.512 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:07.508+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:07.576 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:07.571+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:08.066 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:08.061+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:08.173 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:08.168+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:08.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:08.330+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:08.367 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:08.362+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:08.483 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:08.478+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:08.510 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:08.506+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:08.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:08.573+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:09.081 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:09.075+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:09.150 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:09.145+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:09.318 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:09.314+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:09.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:09.333+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:09.443 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:09.436+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:09.444 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:09.439+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:09.496 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:09.491+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:09.584 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:09.579+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:10.119 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:10.114+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:10.151 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:10.146+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:10.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:10.357+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:10.376 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:10.372+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:10.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:10.464+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:10.485 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:10.476+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:10.485 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:10.479+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:10.604 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:10.597+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:11.125 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:11.120+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:11.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:11.138+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:11.383 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:11.378+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:11.384 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:11.379+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:11.472 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:11.467+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:11.475 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:11.472+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:11.522 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:11.518+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:11.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:11.612+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:12.124 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:12.119+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:12.162 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:12.156+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:12.387 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:12.383+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:12.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:12.422+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:12.434 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:12.427+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:12.435 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:12.430+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:12.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:12.570+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:12.578 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:12.573+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:13.174 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:13.169+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:13.194 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:13.189+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:13.417 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:13.412+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:13.420 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:13.415+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:13.424 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:13.420+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:13.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:13.442+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:13.544 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:13.540+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:13.579 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:13.574+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:14.152 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:14.147+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:14.157 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:14.152+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:14.417 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:14.412+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:14.433 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:14.428+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:14.468 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:14.463+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:14.472 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:14.468+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:14.560 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:14.554+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:14.562 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:14.558+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:15.165 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:15.160+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:15.213 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:15.208+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:15.408 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:15.403+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:15.458 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:15.452+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:15.510 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:15.505+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:15.511 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:15.506+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:15.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:15.516+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:15.583 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:15.578+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:16.173 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:16.168+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:16.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:16.217+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:16.454 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:16.449+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:16.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:16.457+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:16.518 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:16.513+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:16.537 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:16.530+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:16.537 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:16.533+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:16.627 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:16.623+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:16.865 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:23:08.715 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:16.886 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:23:08.736 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:16.934 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:23:08.759 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:16.937 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:23:08.762 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:16.940 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:23:08.765 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:16.940 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:23:08.766 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:17.049 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:23:08.874 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:17.062 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:23:08.887 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:17.077 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:23:08.927 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:44:17.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:17.134+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:17.195 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:17.190+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:17.433 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:17.430+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:17.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:17.457+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:17.510 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:17.506+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:17.520 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:17.513+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:17.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:17.509+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:17.586 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:17.583+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:18.146 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:18.141+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:18.245 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:18.241+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:18.423 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:18.419+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:18.457 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:18.452+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:18.495 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:18.489+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:18.496 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:18.492+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:18.544 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:18.535+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:18.616 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:18.613+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:19.169 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:19.148+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:19.240 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:19.234+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:19.446 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:19.442+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:19.473 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:19.468+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:19.503 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:19.498+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:19.504 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:19.500+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:19.554 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:19.551+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:19.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:19.591+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:20.155 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:20.149+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:20.230 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:20.225+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:20.485 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:20.479+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:20.486 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:20.481+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:20.494 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:20.490+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:20.505 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:20.500+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:20.564 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:20.560+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:20.570 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:20.564+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:21.190 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:21.184+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:21.193 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:21.187+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:21.449 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:21.444+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:21.450 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:21.445+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:21.454 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:21.450+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:21.531 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:21.520+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:21.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:21.539+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:21.559 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:21.555+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:22.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:22.146+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:22.233 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:22.217+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:22.445 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:22.436+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:22.469 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:22.461+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:22.469 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:22.463+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:22.483 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:22.478+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:22.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:22.546+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:22.580 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:22.575+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:23.182 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:23.177+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:23.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:23.193+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:23.408 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:23.400+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:23.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:23.474+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:23.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:23.476+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:23.516 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:23.511+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:23.541 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:23.533+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:23.618 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:23.612+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:24.185 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:24.179+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:24.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:24.233+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:24.445 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:24.442+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:24.469 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:24.463+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:24.472 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:24.468+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:24.495 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:24.490+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:24.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:24.562+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:24.599 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:24.595+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:25.188 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:25.183+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:25.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:25.191+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:25.463 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:25.459+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:25.467 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:25.463+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:25.485 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:25.479+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:25.486 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:25.482+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:25.548 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:25.545+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:25.592 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:25.588+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:26.245 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:26.240+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:26.248 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:26.244+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:26.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:26.449+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:26.498 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:26.491+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:26.521 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:26.516+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:26.523 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:26.520+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:26.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:26.545+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:26.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:26.637+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:26.820 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:44:26.815+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e375 v375) 2019-12-09T19:44:27.219 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:27.213+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:27.249 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:27.236+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:27.408 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:27.403+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:27.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:27.504+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:27.512 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:27.509+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:27.520 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:27.516+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:27.529 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:27.524+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:27.609 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:27.604+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:28.235 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:28.231+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:28.285 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:28.280+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:28.380 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:28.373+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:28.472 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:28.466+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:28.473 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:28.468+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:28.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:28.543+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:28.553 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:28.545+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:28.606 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:28.603+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:29.267 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:29.262+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:29.286 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:29.280+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:29.414 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:29.409+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:29.500 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:29.495+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:29.513 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:29.508+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:29.516 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:29.513+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:29.570 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:29.565+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:29.576 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:29.572+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:30.251 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:30.245+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:30.309 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:30.304+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:30.449 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:30.436+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:30.511 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:30.505+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:30.567 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:30.562+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:30.568 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:30.564+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:30.592 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:30.586+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:30.626 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:30.620+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:31.226 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:31.220+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:31.339 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:31.332+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:31.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:31.423+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:31.520 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:31.515+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:31.566 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:31.562+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:31.572 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:31.567+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:31.601 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:31.590+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:31.643 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:31.638+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:31.836 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:44:31.832+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e375 v375) 2019-12-09T19:44:32.239 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:32.233+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:32.337 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:32.329+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:32.436 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:32.431+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:32.498 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:32.492+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:32.598 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:32.592+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:32.615 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:32.610+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:32.617 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:32.614+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:32.636 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:32.632+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:33.219 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:33.214+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:33.292 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:33.284+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:33.451 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:33.445+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:33.471 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:33.465+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:33.606 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:33.600+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:33.610 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:33.602+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:33.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:33.628+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:33.644 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:33.639+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:34.267 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:34.263+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:34.330 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:34.325+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:34.466 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:34.460+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:34.478 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:34.473+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:34.575 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:34.569+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:34.576 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:34.571+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:34.608 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:34.603+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:34.650 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:34.644+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:35.237 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:35.232+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:35.335 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:35.330+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:35.499 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:35.494+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:35.509 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:35.504+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:35.589 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:35.583+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:35.589 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:35.585+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:35.626 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:35.620+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:35.631 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:35.626+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:36.249 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:36.243+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:36.326 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:36.321+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:36.453 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:36.448+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:36.526 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:36.519+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:36.598 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:36.592+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:36.598 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:36.591+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:36.599 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:36.593+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:36.674 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:36.668+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:36.865 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:44:36.849+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e375 v375) 2019-12-09T19:44:37.212 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:37.204+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:37.304 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:37.299+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:37.499 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:37.493+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:37.512 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:37.507+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:37.577 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:37.568+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:37.579 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:37.574+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:37.631 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:37.627+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:37.677 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:37.672+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:38.203 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:38.198+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:38.283 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:38.278+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:38.473 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:38.468+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:38.522 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:38.515+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:38.564 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:38.559+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:38.567 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:38.563+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:38.630 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:38.625+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:38.671 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:38.665+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:39.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:39.174+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:39.278 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:39.274+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:39.500 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:39.495+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:39.512 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:39.507+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:39.528 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:39.523+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:39.529 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:39.524+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:39.603 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:39.599+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:39.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:39.664+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:40.222 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:40.217+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:40.262 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:40.256+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:40.477 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:40.471+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:40.486 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:40.480+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:40.487 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:40.482+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:40.540 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:40.536+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:40.574 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:40.568+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:40.691 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:40.680+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:41.231 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:44:41.225+0000 1694f700 -1 osd.7 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.627516+0000 front 2019-12-09T19:43:13.669121+0000 (oldest deadline 2019-12-09T19:43:59.494328+0000) 2019-12-09T19:44:41.274 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:44:41.269+0000 1694f700 -1 osd.5 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:10.022344+0000 front 2019-12-09T19:43:09.610415+0000 (oldest deadline 2019-12-09T19:43:55.480814+0000) 2019-12-09T19:44:41.431 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:44:41.427+0000 1694f700 -1 osd.1 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.763345+0000 front 2019-12-09T19:43:11.303524+0000 (oldest deadline 2019-12-09T19:43:57.070543+0000) 2019-12-09T19:44:41.507 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:44:41.502+0000 1694f700 -1 osd.6 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.412856+0000 front 2019-12-09T19:43:13.426927+0000 (oldest deadline 2019-12-09T19:43:57.493977+0000) 2019-12-09T19:44:41.509 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:41.504+0000 1694f700 -1 osd.4 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:43:29.337842+0000 (oldest deadline 2019-12-09T19:44:09.337842+0000) 2019-12-09T19:44:41.512 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:44:41.508+0000 1694f700 -1 osd.4 375 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4385.0:117357 7.4 7.e79ee5c4 (undecoded) ondisk+retry+read+known_if_redirected e373) 2019-12-09T19:44:41.611 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:44:41.607+0000 1694f700 -1 osd.0 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:11.893265+0000 front 2019-12-09T19:43:11.779624+0000 (oldest deadline 2019-12-09T19:43:54.339480+0000) 2019-12-09T19:44:41.712 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:44:41.708+0000 1694f700 -1 osd.2 375 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:43:13.729759+0000 front 2019-12-09T19:43:13.731247+0000 (oldest deadline 2019-12-09T19:43:55.943122+0000) 2019-12-09T19:44:41.949 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:44:41.938+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e375 v375) 2019-12-09T19:45:16.603 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:16.452+0000 34f8d700 -1 osd.3 380 build_incremental_map_msg missing incremental map 380 2019-12-09T19:45:16.702 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:16.641+0000 31f87700 -1 osd.3 380 build_incremental_map_msg missing incremental map 380 2019-12-09T19:45:16.704 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:16.652+0000 34f8d700 -1 osd.3 380 build_incremental_map_msg missing incremental map 380 2019-12-09T19:45:16.925 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:16.840+0000 36790700 -1 osd.3 380 build_incremental_map_msg missing incremental map 380 2019-12-09T19:45:17.100 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.050+0000 34f8d700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:17.179 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.065+0000 34f8d700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:17.205 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.123+0000 34f8d700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:17.308 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.270+0000 31f87700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:17.378 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.311+0000 31786700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:17.533 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:17.511+0000 32f89700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:18.141 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:18.131+0000 3378a700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:45:18.840 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:18.819+0000 37f93700 -1 osd.3 381 build_incremental_map_msg missing incremental map 380 2019-12-09T19:45:18.840 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:45:18.820+0000 37f93700 -1 osd.3 381 build_incremental_map_msg missing incremental map 381 2019-12-09T19:46:15.544 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:15.539+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:15.765 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:15.760+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:16.382 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:16.375+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:16.451 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:16.447+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:16.556 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:16.552+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:16.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:16.798+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:17.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:17.132+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:17.389 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:17.386+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:17.425 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:17.421+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:17.527 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:17.523+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:17.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:17.807+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:18.141 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:18.137+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:18.148 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:18.143+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:18.387 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:18.383+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:18.467 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:18.463+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:18.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:18.510+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:18.798 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:18.793+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:19.108 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:19.103+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:19.182 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:19.178+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:19.395 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:19.391+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:19.396 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:19.393+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:19.480 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:19.472+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:19.552 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:19.543+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:19.836 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:19.832+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:20.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:20.139+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:20.183 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:20.178+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:20.394 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:20.391+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:20.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:20.433+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:20.449 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:20.445+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:20.508 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:20.503+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:20.808 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:20.804+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:21.154 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:21.148+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:21.161 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:21.158+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:21.382 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:21.377+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:21.451 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:21.448+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:21.490 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:21.485+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:21.500 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:21.496+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:21.781 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:21.776+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:22.126 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:22.123+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:22.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:22.153+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:22.425 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:22.421+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:22.444 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:22.440+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:22.447 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:22.444+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:22.548 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:22.543+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:22.759 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:22.754+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:23.135 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:23.130+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:23.168 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:23.164+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:23.417 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:23.413+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:23.418 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:23.414+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:23.439 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:23.435+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:23.569 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:23.565+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:23.784 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:23.779+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:24.185 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:24.181+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:24.200 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:24.196+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:24.410 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:24.407+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:24.417 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:24.414+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:24.433 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:24.430+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:24.609 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:24.605+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:24.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:24.817+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:25.160 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:25.154+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:25.186 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:25.182+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:25.381 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:25.377+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:25.407 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:25.403+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:25.467 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:25.464+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:25.565 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:25.561+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:25.836 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:25.831+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:26.132 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:26.128+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:26.189 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:26.185+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:26.348 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:26.344+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:26.439 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:26.435+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:26.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:26.504+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:26.581 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:26.572+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:26.853 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:26.849+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:27.107 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:27.102+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:27.178 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:27.174+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:27.334 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:27.330+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:27.423 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:27.417+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:27.547 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:27.543+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:27.623 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:27.619+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:27.892 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:27.888+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:28.067 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:28.063+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:28.185 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:28.181+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:28.324 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:28.320+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:28.425 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:28.420+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:28.550 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:28.544+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:28.581 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:28.576+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:28.890 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:28.885+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:29.056 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:29.052+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:29.159 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:29.155+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:29.311 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:29.307+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:29.382 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:29.378+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:29.507 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:29.503+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:29.591 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:29.588+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:29.842 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:29.838+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:30.109 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:30.104+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:30.175 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:30.171+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:30.297 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:30.292+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:30.424 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:30.417+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:30.513 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:30.508+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:30.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:30.606+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:30.883 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:30.873+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:31.104 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:31.099+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:31.154 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:31.149+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:31.313 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:31.308+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:31.470 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:31.465+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:31.511 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:31.506+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:31.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:31.644+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:31.926 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:31.919+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:32.122 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:32.118+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:32.196 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:32.191+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:32.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:32.289+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:32.471 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:32.466+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:32.545 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:32.540+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:32.621 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:32.615+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:32.936 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:32.931+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:33.162 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:33.157+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:33.218 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:33.213+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:33.267 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:33.263+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:33.426 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:33.422+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:33.521 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:33.516+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:33.669 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:33.664+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:33.967 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:33.961+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:34.172 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:34.168+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:34.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:34.255+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:34.270 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:34.265+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:34.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:34.396+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:34.536 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:34.532+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:34.696 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:34.691+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:34.992 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:34.988+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:35.217 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:35.211+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:35.261 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:35.257+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:35.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:35.285+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:35.399 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:35.388+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:35.562 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:35.558+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:35.653 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:35.646+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:35.968 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:35.964+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:36.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:36.202+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:36.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:36.224+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:36.269 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:36.265+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:36.362 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:36.358+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:36.604 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:36.598+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:36.626 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:36.617+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:36.973 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:36.969+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:37.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:37.158+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:37.280 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:37.272+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:37.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:37.306+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:37.371 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:37.367+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:37.607 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:37.604+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:37.645 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:37.642+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:38.010 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:38.005+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:38.122 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:38.119+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:38.260 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:38.255+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:38.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:38.306+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:38.356 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:38.350+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:38.612 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:38.606+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:38.615 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:38.610+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:39.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:39.038+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:39.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:39.116+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:39.251 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:39.247+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:39.288 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:39.283+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:39.402 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:39.397+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:39.597 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:39.593+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:39.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:39.644+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:40.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:40.036+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:40.109 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:40.104+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:40.231 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:40.227+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:40.245 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:40.242+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:40.397 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:40.393+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:40.569 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:40.563+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:40.665 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:40.659+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:41.071 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:41.066+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:41.096 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:41.091+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:41.218 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:41.212+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:41.241 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:41.235+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:41.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:41.398+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:41.608 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:41.603+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:41.636 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:41.631+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:42.049 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:42.043+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:42.112 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:42.108+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:42.232 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:42.227+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:42.235 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:42.231+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:42.446 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:42.442+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:42.633 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:42.629+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:42.637 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:42.634+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:43.013 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:43.009+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:43.091 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:43.087+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:43.236 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:43.232+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:43.257 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:43.248+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:43.436 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:43.431+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:43.594 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:43.590+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:43.679 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:43.674+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:44.024 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:44.014+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:44.054 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:44.049+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:44.233 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:44.229+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:44.271 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:44.263+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:44.431 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:44.426+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:44.618 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:44.615+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:44.645 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:44.638+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:44.981 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:44.977+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:45.067 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:45.063+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:45.196 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:45.191+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:45.233 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:45.227+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:45.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:45.432+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:45.629 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:45.626+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:45.648 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:25:37.498 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.661 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:25:37.512 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.671 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:25:37.496 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.673 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:25:37.498 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.677 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:25:37.502 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.679 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:25:37.504 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.691 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:45.686+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:45.834 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:25:37.659 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.860 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:25:37.710 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.888 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:25:37.713 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:46:45.980 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:45.976+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:46.091 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:46.088+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:46.195 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:46.192+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:46.242 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:46.238+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:46.408 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:46.404+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:46.597 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:46.592+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:46.731 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:46.725+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:46.980 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:46.976+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:47.128 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:47.123+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:47.186 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:47.181+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:47.249 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:47.245+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:47.411 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:47.408+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:47.415 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:46:47.410+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e381 v381) 2019-12-09T19:46:47.641 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:47.638+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:47.742 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:47.737+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:47.944 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:47.939+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:48.154 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:48.149+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:48.198 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:48.193+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:48.207 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:48.203+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:48.403 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:48.398+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:48.694 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:48.689+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:48.760 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:48.755+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:48.951 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:48.947+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:49.118 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:49.113+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:49.171 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:49.165+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:49.258 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:49.254+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:49.448 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:49.443+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:49.651 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:49.647+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:49.719 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:49.715+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:49.996 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:49.992+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:50.113 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:50.108+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:50.222 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:50.218+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:50.253 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:50.249+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:50.426 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:50.421+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:50.717 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:50.700+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:50.735 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:50.731+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:51.011 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:51.004+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:51.136 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:51.131+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:51.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:51.222+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:51.241 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:51.237+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:51.383 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:51.379+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:51.699 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:46:51.695+0000 1694f700 -1 osd.3 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.850035+0000 front 2019-12-09T19:45:34.135540+0000 (oldest deadline 2019-12-09T19:46:18.981152+0000) 2019-12-09T19:46:51.771 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:46:51.766+0000 1694f700 -1 osd.7 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:35.434198+0000 front 2019-12-09T19:45:32.211007+0000 (oldest deadline 2019-12-09T19:46:14.680204+0000) 2019-12-09T19:46:52.060 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:46:52.055+0000 1694f700 -1 osd.6 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.858002+0000 front 2019-12-09T19:45:32.866131+0000 (oldest deadline 2019-12-09T19:46:16.277011+0000) 2019-12-09T19:46:52.101 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:46:52.095+0000 1694f700 -1 osd.5 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.862741+0000 front 2019-12-09T19:45:33.284445+0000 (oldest deadline 2019-12-09T19:46:15.495367+0000) 2019-12-09T19:46:52.242 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:46:52.238+0000 1694f700 -1 osd.1 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.849803+0000 front 2019-12-09T19:45:33.475374+0000 (oldest deadline 2019-12-09T19:46:15.812411+0000) 2019-12-09T19:46:52.257 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:46:52.254+0000 1694f700 -1 osd.2 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:32.989433+0000 front 2019-12-09T19:45:33.499560+0000 (oldest deadline 2019-12-09T19:46:17.245584+0000) 2019-12-09T19:46:52.428 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:46:52.424+0000 1694f700 -1 osd.0 381 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:45:34.344582+0000 front 2019-12-09T19:45:32.971236+0000 (oldest deadline 2019-12-09T19:46:15.782797+0000) 2019-12-09T19:46:52.499 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:46:52.490+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e381 v381) 2019-12-09T19:47:29.705 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:47:29.576+0000 37792700 -1 osd.3 385 build_incremental_map_msg missing incremental map 385 2019-12-09T19:47:30.024 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:47:29.739+0000 32788700 -1 osd.3 385 build_incremental_map_msg missing incremental map 385 2019-12-09T19:47:30.033 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:47:29.803+0000 3378a700 -1 osd.3 385 build_incremental_map_msg missing incremental map 385 2019-12-09T19:47:31.647 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.607+0000 35f8f700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.751 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.638+0000 38794700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.752 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.639+0000 38794700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.789 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.667+0000 31786700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.817 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.692+0000 32f89700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.819 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.695+0000 114ae700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.721+0000 31786700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.857 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.726+0000 31786700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:31.963 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:31.957+0000 32f89700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:32.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:47:32.394+0000 36f91700 -1 osd.4 387 build_incremental_map_msg missing incremental map 387 2019-12-09T19:47:51.880 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:51.875+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:52.381 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:52.375+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:52.689 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:52.684+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:52.758 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:52.753+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:52.914 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:52.909+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:53.100 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:26:44.950 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.141 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:26:44.991 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.161 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:26:44.986 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.162 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:26:44.987 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.164 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:26:44.989 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.180 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:26:45.005 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.236 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:26:45.086 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.265 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:26:45.090 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.292 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:26:45.117 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:47:53.295 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:53.292+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:53.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:53.349+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:53.719 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:53.715+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:53.753 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:53.749+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:53.911 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:53.907+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:54.310 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:54.306+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:54.311 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:54.307+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:54.772 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:54.767+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:54.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:54.771+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:54.943 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:54.938+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:55.275 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:55.270+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:55.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:55.287+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:55.756 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:55.751+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:55.763 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:55.758+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:55.972 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:55.967+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:56.283 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:56.278+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:56.295 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:56.289+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:56.766 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:56.761+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:56.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:56.763+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:56.980 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:56.976+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:57.270 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:57.266+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:57.322 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:57.317+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:57.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:57.782+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:57.825 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:57.819+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:57.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:57.973+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:58.297 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:58.292+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:58.362 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:58.358+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:58.762 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:58.757+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:58.779 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:58.774+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:58.955 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:58.951+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:47:59.324 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:47:59.319+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:47:59.336 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:47:59.331+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:47:59.753 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:47:59.746+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:47:59.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:47:59.792+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:47:59.966 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:47:59.961+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:00.306 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:00.303+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:00.360 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:00.355+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:00.722 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:00.708+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:00.801 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:00.795+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:00.965 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:00.960+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:01.311 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:01.307+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:01.317 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:01.313+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:01.680 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:01.675+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:01.806 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:01.801+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:01.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:01.943+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:02.294 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:02.291+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:02.300 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:02.295+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:02.481 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:02.475+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:02.672 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:02.668+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:02.782 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:02.779+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:02.934 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:02.930+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:03.248 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:03.245+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:03.327 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:03.324+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:03.534 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:03.531+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:03.705 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:03.701+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:03.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:03.792+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:03.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:03.896+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:04.262 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:04.258+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:04.367 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:04.362+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:04.506 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:04.503+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:04.691 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:04.687+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:04.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:04.784+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:04.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:04.918+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:05.275 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:05.271+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:05.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:05.335+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:05.544 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:05.539+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:05.723 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:05.718+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:05.826 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:05.821+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:05.950 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:05.945+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:06.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:06.305+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:06.332 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:06.325+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:06.547 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:06.543+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:06.710 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:06.704+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:06.867 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:06.862+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:06.976 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:06.971+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:07.266 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:07.261+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:07.286 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:07.281+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:07.505 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:07.500+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:07.509 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:07.505+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:07.741 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:07.735+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:07.871 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:07.866+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:08.018 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:08.015+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:08.240 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:08.236+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:08.331 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:08.327+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:08.467 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:08.463+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:08.471 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:08.467+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:08.726 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:08.721+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:08.836 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:08.830+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:09.035 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:09.030+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:09.243 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:09.240+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:09.305 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:09.299+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:09.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:09.515+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:09.522 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:09.517+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:09.729 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:09.724+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:09.805 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:09.800+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:10.070 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:10.066+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:10.225 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:10.221+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:10.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:10.272+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:10.555 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:10.550+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:10.555 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:10.552+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:10.733 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:10.727+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:10.766 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:10.761+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:11.104 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:11.100+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:11.204 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:11.201+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:11.308 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:11.303+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:11.544 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:11.538+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:11.545 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:11.541+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:11.743 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:11.738+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:11.752 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:11.748+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:12.122 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:12.118+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:12.190 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:12.187+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:12.302 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:12.297+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:12.572 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:12.566+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:12.572 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:12.568+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:12.787 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:12.781+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:12.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:12.782+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:13.133 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:13.128+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:13.168 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:13.164+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:13.323 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:13.317+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:13.541 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:13.536+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:13.543 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:13.538+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:13.783 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:13.778+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:13.814 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:13.808+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:14.112 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:14.108+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:14.215 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:14.211+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:14.365 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:14.361+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:14.508 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:14.503+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:14.509 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:14.505+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:14.767 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:14.762+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:14.780 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:14.775+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:15.080 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:15.075+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:15.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:15.255+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:15.349 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:15.345+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:15.475 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:15.469+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:15.476 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:15.471+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:15.744 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:15.740+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:15.825 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:15.820+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:16.137 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:16.126+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:16.291 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:16.286+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:16.335 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:16.330+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:16.483 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:16.479+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:16.486 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:16.483+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:16.804 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:16.798+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:16.824 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:16.819+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:17.115 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:17.110+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:17.341 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:17.335+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:17.345 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:17.340+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:17.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:17.514+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:17.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:17.516+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:17.843 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:17.837+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:17.852 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:17.848+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:18.117 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:18.111+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:18.328 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:18.323+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:18.337 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:18.332+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:18.563 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:18.559+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:18.565 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:18.561+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:18.843 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:18.838+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:18.848 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:18.844+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:19.130 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:19.125+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:19.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:19.313+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:19.356 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:19.350+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:19.593 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:19.587+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:19.593 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:19.590+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:19.845 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:19.840+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:19.901 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:19.897+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:20.157 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:20.152+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:20.307 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:20.299+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:20.318 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:20.312+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:20.628 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:20.622+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:20.629 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:20.625+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:20.859 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:20.854+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:20.955 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:20.949+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:21.168 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:21.163+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:21.298 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:21.293+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:21.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:21.334+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:21.680 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:21.672+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:21.681 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:21.675+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:21.869 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:21.864+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:21.966 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:21.962+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:22.132 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:22.128+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:22.276 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:22.269+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:22.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:22.309+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:22.648 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:22.641+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:22.649 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:22.644+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:22.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:22.870+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:22.979 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:22.975+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:23.136 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:23.131+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:23.164 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:48:23.157+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e387 v387) 2019-12-09T19:48:23.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:23.286+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:23.330 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:23.325+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:23.641 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:23.635+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:23.642 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:23.638+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:23.917 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:23.909+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:23.997 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:23.991+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:24.186 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:24.182+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:24.283 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:24.276+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:24.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:24.356+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:24.676 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:24.665+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:24.677 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:24.667+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:24.909 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:24.903+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:25.029 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:25.024+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:25.180 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:25.175+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:25.251 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:25.246+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:25.337 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:25.331+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:25.676 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:25.672+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:25.680 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:25.676+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:25.869 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:25.864+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:26.034 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:26.029+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:26.141 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:26.135+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:26.241 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:26.235+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:26.345 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:26.341+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:26.654 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:26.649+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:26.657 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:26.653+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:26.911 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:26.906+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:27.023 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:27.019+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:27.104 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:27.099+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:27.234 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:27.230+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:27.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:27.356+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:27.679 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:27.674+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:27.680 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:27.676+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:27.879 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:27.873+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:28.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:28.038+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:28.099 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:28.092+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:28.184 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:48:28.176+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e387 v387) 2019-12-09T19:48:28.281 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:28.277+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:28.374 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:28.368+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:28.664 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:28.658+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:28.665 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:28.660+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:28.906 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:28.900+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:29.022 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:29.017+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:29.113 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:29.108+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:29.339 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:29.330+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:29.416 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:29.411+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:29.627 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:29.621+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:29.627 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:29.624+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:29.905 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:29.900+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:29.994 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:29.988+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:30.103 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:30.098+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:30.379 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:30.375+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:30.434 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:30.429+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:30.587 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:30.582+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:30.588 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:30.584+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:30.869 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:30.864+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:30.958 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:30.952+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:31.086 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:31.081+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:31.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:31.380+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:31.478 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:31.473+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:31.615 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:31.608+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:31.615 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:31.610+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:31.872 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:31.867+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:31.917 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:31.912+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:32.065 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:32.060+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:32.401 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:32.395+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:32.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:32.461+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:32.574 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:32.570+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:32.577 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:32.570+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:32.578 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:32.574+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:32.847 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:32.842+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:32.875 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:32.870+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:33.057 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:33.052+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:33.203 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:48:33.196+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e387 v387) 2019-12-09T19:48:33.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:33.376+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:33.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:33.425+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:33.535 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:33.529+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:33.536 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:33.531+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:33.616 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:33.610+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:33.896 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:33.891+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:33.925 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:33.919+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:34.106 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:34.101+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:34.378 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:34.373+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:34.411 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:34.406+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:34.568 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:34.564+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:34.570 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:34.567+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:34.610 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:34.603+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:34.876 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:34.870+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:34.973 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:34.968+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:35.144 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:35.138+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:35.387 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:35.383+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:35.430 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:35.426+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:35.551 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:35.545+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:35.552 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:35.547+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:35.654 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:35.650+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:35.929 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:35.924+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:36.011 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:36.004+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:36.107 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:36.103+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:36.349 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:36.344+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:36.435 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:36.430+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:36.603 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:36.597+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:36.604 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:36.600+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:36.704 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:36.694+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:36.918 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:36.913+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:36.991 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:36.987+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:37.081 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:37.076+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:37.362 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:37.357+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:37.423 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:37.417+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:37.600 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:37.592+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:37.601 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:37.593+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:37.677 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:37.672+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:37.954 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:37.950+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:38.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:38.002+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:38.085 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:38.080+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:38.222 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:48:38.214+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e387 v387) 2019-12-09T19:48:38.363 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:38.358+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:38.420 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:38.416+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:38.565 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:38.561+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:38.566 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:38.562+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:38.690 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:38.686+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:38.955 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:38.950+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:39.034 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:39.029+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:39.089 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:39.084+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:39.332 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:39.329+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:39.407 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:39.402+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:39.580 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:39.574+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:39.581 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:39.576+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:39.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:39.664+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:39.943 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:39.937+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:40.058 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:40.054+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:40.103 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:40.099+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:40.298 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:40.294+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:40.371 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:40.366+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:40.550 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:40.544+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:40.551 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:40.546+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:40.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:40.641+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:40.910 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:40.905+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:41.076 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:41.070+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:41.077 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:41.073+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:41.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:41.258+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:41.343 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:41.338+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:41.532 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:41.527+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:41.536 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:41.532+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:41.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:41.634+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:41.872 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:41.866+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:42.075 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:42.070+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:42.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:42.105+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:42.286 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:42.281+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:42.304 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:42.299+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:42.575 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:42.569+0000 1694f700 -1 osd.4 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:47:26.664000+0000 (oldest deadline 2019-12-09T19:48:06.664000+0000) 2019-12-09T19:48:42.576 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:42.572+0000 1694f700 -1 osd.4 387 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:48:42.605 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:42.600+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:42.908 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:48:42.903+0000 1694f700 -1 osd.7 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.452221+0000 front 2019-12-09T19:47:07.590147+0000 (oldest deadline 2019-12-09T19:47:52.062749+0000) 2019-12-09T19:48:43.093 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:48:43.088+0000 1694f700 -1 osd.2 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:05.924985+0000 front 2019-12-09T19:47:05.925863+0000 (oldest deadline 2019-12-09T19:47:51.187431+0000) 2019-12-09T19:48:43.149 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:48:43.144+0000 1694f700 -1 osd.5 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.587712+0000 front 2019-12-09T19:47:07.301376+0000 (oldest deadline 2019-12-09T19:47:52.360762+0000) 2019-12-09T19:48:43.311 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:48:43.304+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e387 v387) 2019-12-09T19:48:43.320 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:48:43.315+0000 1694f700 -1 osd.1 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:07.965308+0000 front 2019-12-09T19:47:08.187726+0000 (oldest deadline 2019-12-09T19:47:53.047927+0000) 2019-12-09T19:48:43.339 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:48:43.335+0000 1694f700 -1 osd.6 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:06.141577+0000 front 2019-12-09T19:47:06.199891+0000 (oldest deadline 2019-12-09T19:47:51.940923+0000) 2019-12-09T19:48:43.640 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:48:43.637+0000 1694f700 -1 osd.0 387 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:47:09.357792+0000 front 2019-12-09T19:47:08.787224+0000 (oldest deadline 2019-12-09T19:48:31.685653+0000) 2019-12-09T19:48:43.726 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:48:43.633+0000 1694f700 -1 osd.4 388 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e385) 2019-12-09T19:50:12.519 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:12.516+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:12.619 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:12.616+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:12.725 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:12.716+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:13.061 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:13.056+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:13.320 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:13.315+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:13.544 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:13.538+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:13.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:13.602+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:13.706 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:13.703+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:13.755 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:13.752+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:14.044 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:14.036+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:14.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:14.300+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:14.502 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:14.498+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:14.551 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:14.546+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:14.631 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:14.627+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:14.708 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:14.703+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:14.746 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:14.743+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:15.025 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:15.021+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:15.346 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:15.342+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:15.522 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:15.517+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:15.573 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:15.568+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:15.654 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:15.649+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:15.748 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:15.743+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:15.751 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:15.747+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:16.038 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:16.034+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:16.338 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:16.333+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:16.555 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:16.550+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:16.570 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:16.566+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:16.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:16.644+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:16.770 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:16.765+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:16.791 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:16.786+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:17.047 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:17.043+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:17.298 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:17.295+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:17.520 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:17.515+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:17.617 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:17.614+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:17.661 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:17.655+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:17.748 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:17.745+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:17.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:17.792+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:18.078 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:18.075+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:18.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:18.297+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:18.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:18.537+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:18.609 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:18.605+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:18.616 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:18.612+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:18.714 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:18.711+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:18.785 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:18.781+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:19.076 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:19.072+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:19.321 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:19.316+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:19.565 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:19.561+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:19.569 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:19.563+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:19.616 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:19.612+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:19.712 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:19.708+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:19.809 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:19.804+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:20.106 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:20.101+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:20.373 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:20.370+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:20.553 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:20.550+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:20.565 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:20.558+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:20.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:20.599+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:20.755 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:20.751+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:20.778 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:20.773+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:21.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:21.056+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:21.427 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:21.422+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:21.556 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:21.552+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:21.592 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:21.587+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:21.613 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:21.606+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:21.746 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:21.741+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:21.752 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:21.748+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:22.092 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:22.088+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:22.442 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:22.438+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:22.539 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:22.534+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:22.601 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:22.597+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:22.638 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:22.633+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:22.733 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:22.728+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:22.774 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:22.770+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:23.086 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:23.082+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:23.439 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:23.435+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:23.503 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:23.498+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:23.595 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:23.591+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:23.596 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:23.593+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:23.690 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:23.686+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:23.769 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:23.765+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:24.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:24.086+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:24.420 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:24.417+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:24.474 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:24.470+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:24.576 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:24.572+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:24.594 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:24.591+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:24.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:24.685+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:24.765 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:24.761+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:25.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:25.092+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:25.384 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:25.379+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:25.439 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:25.434+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:25.587 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:25.581+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:25.601 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:25.597+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:25.740 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:25.736+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:25.814 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:25.810+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:26.061 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:26.056+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:26.363 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:26.358+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:26.419 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:26.411+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:26.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:26.568+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:26.598 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:26.594+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:26.742 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:26.738+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:26.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:26.785+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:27.026 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:27.022+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:27.332 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:27.328+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:27.408 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:27.404+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:27.547 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:27.539+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:27.646 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:27.643+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:27.771 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:27.766+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:27.784 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:27.781+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:28.062 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:28.056+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:28.334 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:28.329+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:28.430 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:28.426+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:28.574 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:28.570+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:28.617 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:28.612+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:28.734 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:28.729+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:28.809 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:28.803+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:29.085 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:29.080+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:29.387 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:29.382+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:29.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:29.471+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:29.561 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:29.557+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:29.642 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:29.639+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:29.728 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:29.724+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:29.815 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:29.811+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:30.080 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:30.077+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:30.349 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:30.345+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:30.459 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:30.454+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:30.577 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:30.572+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:30.637 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:30.633+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:30.720 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:30.715+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:30.784 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:30.778+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:31.097 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:31.093+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:31.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:31.357+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:31.443 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:31.437+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:31.617 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:31.613+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:31.634 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:31.629+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:31.706 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:31.701+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:31.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:31.791+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:32.129 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:32.126+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:32.389 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:32.384+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:32.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:32.425+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:32.597 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:32.593+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:32.640 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:32.636+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:32.745 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:32.740+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:32.804 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:32.799+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:33.138 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:33.134+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:33.425 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:33.420+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:33.434 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:33.430+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:33.555 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:33.552+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:33.628 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:33.623+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:33.728 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:33.724+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:33.786 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:33.781+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:34.104 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:34.100+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:34.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:34.442+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:34.478 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:34.474+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:34.564 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:34.558+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:34.621 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:34.616+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:34.744 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:34.741+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:34.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:34.783+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:35.127 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:35.122+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:35.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:35.444+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:35.492 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:35.488+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:35.579 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:35.575+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:35.596 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:35.592+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:35.705 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:35.701+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:35.774 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:35.771+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:36.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:36.094+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:36.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:36.423+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:36.458 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:36.454+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:36.565 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:36.560+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:36.579 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:36.574+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:36.735 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:36.731+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:36.739 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:36.734+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:37.123 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:37.119+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:37.403 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:37.399+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:37.504 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:37.500+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:37.546 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:37.541+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:37.606 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:37.602+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:37.750 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:37.746+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:37.779 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:37.775+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:38.169 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:38.165+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:38.420 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:38.410+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:38.548 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:38.544+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:38.594 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:38.590+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:38.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:38.606+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:38.801 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:38.797+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:38.804 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:38.800+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:39.177 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:39.172+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:39.470 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:39.466+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:39.570 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:39.566+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:39.582 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:39.577+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:39.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:39.598+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:39.782 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:39.776+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:39.813 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:39.809+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:40.215 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:29:32.065 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:40.222+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:40.238 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:29:32.088 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.241 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:29:32.066 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.243 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:29:32.068 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.249 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:29:32.074 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.250 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:29:32.075 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.437 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:29:32.287 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.445 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:29:32.270 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:40.454+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:40.503 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:29:32.328 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:50:40.558 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:40.555+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:40.582 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:40.579+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:40.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:40.606+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:40.755 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:40.751+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:40.787 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:40.783+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:41.280 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:41.275+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:41.424 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:41.421+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:41.528 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:41.523+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:41.555 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:41.551+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:41.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:41.600+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:41.808 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:41.804+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:41.831 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:41.827+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:42.327 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:42.323+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:42.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:42.441+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:42.536 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:42.531+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:42.555 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:42.552+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:42.617 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:42.613+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:42.765 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:42.762+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:42.815 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:42.812+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:43.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:43.336+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:43.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:43.470+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:43.493 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:43.489+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:43.547 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:43.543+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:43.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:43.599+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:43.725 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:43.722+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:43.765 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:50:43.758+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 47sec e392 v392) 2019-12-09T19:50:43.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:43.771+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:44.362 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:44.353+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:44.431 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:44.426+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:44.539 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:44.536+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:44.565 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:44.560+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:44.633 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:44.626+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:44.713 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:44.708+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:44.797 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:44.792+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:45.410 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:45.406+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:45.479 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:45.474+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:45.537 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:45.533+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:45.579 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:45.574+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:45.639 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:45.634+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:45.699 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:45.693+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:45.839 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:45.836+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:46.373 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:46.369+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:46.501 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:46.497+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:46.556 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:46.551+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:46.562 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:46.557+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:46.610 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:46.605+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:46.685 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:46.681+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:46.866 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:46.862+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:47.412 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:47.407+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:47.526 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:47.517+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:47.545 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:47.532+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:47.569 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:47.563+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:47.639 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:47.636+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:47.643 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:47.638+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:47.842 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:47.838+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:48.387 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:50:48.383+0000 1694f700 -1 osd.0 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:31.058946+0000 front 2019-12-09T19:49:32.367007+0000 (oldest deadline 2019-12-09T19:50:12.630658+0000) 2019-12-09T19:50:48.495 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:50:48.491+0000 1694f700 -1 osd.6 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:28.507042+0000 front 2019-12-09T19:49:28.500471+0000 (oldest deadline 2019-12-09T19:50:13.788677+0000) 2019-12-09T19:50:48.496 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:50:48.490+0000 1694f700 -1 osd.3 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.331503+0000 front 2019-12-09T19:49:27.299219+0000 (oldest deadline 2019-12-09T19:50:12.155271+0000) 2019-12-09T19:50:48.547 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:50:48.542+0000 1694f700 -1 osd.2 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:26.954415+0000 front 2019-12-09T19:49:27.471571+0000 (oldest deadline 2019-12-09T19:50:12.594294+0000) 2019-12-09T19:50:48.594 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:50:48.590+0000 1694f700 -1 osd.7 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:29.458543+0000 front 2019-12-09T19:49:29.580505+0000 (oldest deadline 2019-12-09T19:50:12.246524+0000) 2019-12-09T19:50:48.641 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:50:48.636+0000 1694f700 -1 osd.1 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:32.294044+0000 front 2019-12-09T19:49:29.056231+0000 (oldest deadline 2019-12-09T19:50:11.738490+0000) 2019-12-09T19:50:48.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:50:48.817+0000 1694f700 -1 osd.5 392 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:49:30.628468+0000 front 2019-12-09T19:49:30.629230+0000 (oldest deadline 2019-12-09T19:50:13.072789+0000) 2019-12-09T19:50:48.855 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:50:48.848+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 47sec e392 v392) 2019-12-09T19:51:19.520 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:19.516+0000 1694f700 -1 osd.6 394 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:120550 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 821612096 gen 121] snapc 0=[] ondisk+write+known_if_redirected e393) 2019-12-09T19:51:40.727 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:40.588+0000 36790700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.809 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:40.693+0000 35f8f700 -1 osd.6 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.809 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:40.613+0000 11caf700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.904 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:40.687+0000 34f8d700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.911 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:40.845+0000 37f93700 -1 osd.6 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.915 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:40.858+0000 37f93700 -1 osd.6 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.918 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:40.878+0000 124b0700 -1 osd.6 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:40.966 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:40.779+0000 31786700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.008 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:40.951+0000 38f95700 -1 osd.6 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.019 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:40.980+0000 32b88700 -1 osd.1 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.126 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:40.957+0000 38794700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.180 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:51:41.060+0000 32f89700 -1 osd.0 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.202 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:41.131+0000 34b8c700 -1 osd.1 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.299 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:51:41.037+0000 37792700 -1 osd.3 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.301 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:51:41.042+0000 35f8f700 -1 osd.3 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:41.492 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:51:41.385+0000 37792700 -1 osd.3 396 build_incremental_map_msg missing incremental map 396 2019-12-09T19:51:42.175 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.142+0000 31786700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.262 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.166+0000 35f8f700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.344 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.205+0000 36790700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.388 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.225+0000 36790700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.427 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.234+0000 36790700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.429 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.239+0000 36790700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:42.464 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:42.299+0000 35f8f700 -1 osd.4 397 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:43.206 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.161+0000 124b0700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.259 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.198+0000 34f8d700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.302 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.265+0000 30f85700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.312 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.293+0000 30f85700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.511 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.339+0000 31786700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.586 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.500+0000 31786700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:43.911 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.899+0000 30f85700 -1 osd.4 398 build_incremental_map_msg missing incremental map 397 2019-12-09T19:51:43.912 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.899+0000 30f85700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:44.009 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:43.990+0000 31f87700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:44.535 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:51:44.530+0000 32788700 -1 osd.4 398 build_incremental_map_msg missing incremental map 398 2019-12-09T19:51:45.967 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:45.963+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:46.965 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:46.960+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:47.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:47.261+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:47.961 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:47.957+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:48.274 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:48.263+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:48.948 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:48.943+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:49.269 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:49.264+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:49.731 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:49.725+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:49.927 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:49.921+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:50.270 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:50.265+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:50.273 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:50.270+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:50.734 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:50.729+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:50.957 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:50.952+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:51.299 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:51.294+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:51.313 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:51.306+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:51.749 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:51.742+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:51.921 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:51.917+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:51.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:51.988+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:52.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:52.279+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:52.289 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:52.284+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:52.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:52.700+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:52.918 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:52.915+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:53.023 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:53.018+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:53.250 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:53.245+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:53.288 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:53.284+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:53.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:53.722+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:53.933 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:53.926+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:54.060 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:54.054+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:54.218 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:54.214+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:54.278 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:54.272+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:54.781 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:54.771+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:54.898 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:54.890+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:55.013 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:55.009+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:55.248 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:55.243+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:55.275 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:55.270+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:55.805 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:55.802+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:55.901 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:55.897+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:56.012 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:56.007+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:56.261 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:56.255+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:56.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:56.264+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:56.787 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:56.782+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:56.925 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:56.920+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:56.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:56.987+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:57.231 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:57.226+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:57.304 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:57.298+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:57.835 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:57.832+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:57.964 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:57.959+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:57.968 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:57.964+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:58.253 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:58.246+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:58.328 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:58.323+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:58.810 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:58.805+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:58.922 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:58.917+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:58.949 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:58.944+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:51:59.232 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:51:59.226+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:51:59.368 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:51:59.363+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:51:59.824 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:51:59.820+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:51:59.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:51:59.885+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:51:59.992 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:51:59.988+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:00.245 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:00.240+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:00.402 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:00.398+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:00.816 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:00.811+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:00.883 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:00.878+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:00.958 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:00.952+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:01.216 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:01.211+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:01.386 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:01.379+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:01.778 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:01.772+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:01.881 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:01.877+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:01.917 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:01.912+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:02.234 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:02.230+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:02.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:02.358+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:02.743 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:02.738+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:02.855 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:02.851+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:02.967 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:02.963+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:03.231 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:03.225+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:03.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:03.373+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:03.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:03.736+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:03.847 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:03.842+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:03.946 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:03.941+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:04.190 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:04.186+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:04.403 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:04.398+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:04.731 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:04.726+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:04.736 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:04.731+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:04.838 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:04.833+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:04.971 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:04.966+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:05.146 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:05.141+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:05.381 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:05.373+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:05.720 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:05.716+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:05.740 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:05.735+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:05.801 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:05.796+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:05.993 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:05.982+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:06.140 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:06.135+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:06.409 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:06.404+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:06.701 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:06.696+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:06.777 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:06.772+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:06.796 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:06.791+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:07.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:07.002+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:07.161 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:07.156+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:07.426 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:07.415+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:07.728 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:07.723+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:07.805 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:07.800+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:07.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:07.823+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:08.002 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:07.997+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:08.157 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:08.152+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:08.473 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:08.468+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:08.745 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:08.741+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:08.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:08.781+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:08.839 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:08.834+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:09.024 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:09.018+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:09.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:09.132+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:09.493 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:09.488+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:09.785 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:09.779+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:09.806 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:09.801+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:09.839 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:09.833+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:10.013 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:10.008+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:10.146 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:10.140+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:10.547 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:10.542+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:10.737 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:10.732+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:10.816 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:10.811+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:10.848 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:10.842+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:10.973 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:10.967+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:11.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:11.150+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:11.307 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:11.303+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:11.522 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:11.516+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:11.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:11.710+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:11.782 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:11.778+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:11.889 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:11.884+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:11.973 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:11.967+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:12.205 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:12.199+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:12.266 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:12.260+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:12.499 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:12.494+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:12.673 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:12.667+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:12.755 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:12.750+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:12.887 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:12.882+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:12.959 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:12.954+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:13.233 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:13.229+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:13.307 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:13.303+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:13.529 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:13.524+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:13.708 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:13.703+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:13.763 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:13.759+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:13.843 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:13.839+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:13.960 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:13.956+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:14.252 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:14.247+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:14.263 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:14.259+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:14.565 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:14.560+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:14.763 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:14.752+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:14.799 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:14.795+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:14.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:14.795+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:14.995 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:14.990+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:15.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:15.232+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:15.290 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:15.286+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:15.587 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:15.581+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:15.793 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:15.785+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:15.794 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:15.787+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:15.835 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:15.830+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:16.017 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:16.012+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:16.220 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:16.215+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:16.275 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:16.270+0000 1694f700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:51:35.930370+0000 (oldest deadline 2019-12-09T19:52:15.930370+0000) 2019-12-09T19:52:16.278 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:16.274+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:16.566 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:16.560+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:16.778 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:16.774+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:16.842 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:16.838+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:16.875 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:16.870+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:17.044 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:17.039+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:17.258 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:17.252+0000 1694f700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:51:35.930370+0000 (oldest deadline 2019-12-09T19:52:15.930370+0000) 2019-12-09T19:52:17.259 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:17.255+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:17.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:17.259+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:17.541 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:17.537+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:17.761 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:17.753+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:17.823 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:17.806+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:17.901 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:17.895+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:18.020 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:18.015+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:18.227 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:18.221+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:18.237 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:18.230+0000 1694f700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:51:35.930370+0000 (oldest deadline 2019-12-09T19:52:15.930370+0000) 2019-12-09T19:52:18.237 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:18.232+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:18.503 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:52:18.497+0000 1694f700 -1 osd.5 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.476854+0000 front 2019-12-09T19:51:04.329647+0000 (oldest deadline 2019-12-09T19:51:50.210269+0000) 2019-12-09T19:52:18.781 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:18.777+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:18.796 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:52:18.792+0000 1694f700 -1 osd.2 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:04.984526+0000 front 2019-12-09T19:51:04.990258+0000 (oldest deadline 2019-12-09T19:51:49.642921+0000) 2019-12-09T19:52:18.922 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:52:18.916+0000 1694f700 -1 osd.7 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.856767+0000 front 2019-12-09T19:51:01.923145+0000 (oldest deadline 2019-12-09T19:51:45.938968+0000) 2019-12-09T19:52:19.056 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:52:19.051+0000 1694f700 -1 osd.1 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:05.770361+0000 front 2019-12-09T19:51:05.832951+0000 (oldest deadline 2019-12-09T19:51:51.003960+0000) 2019-12-09T19:52:19.250 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:19.241+0000 1694f700 -1 osd.4 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:51:35.930370+0000 (oldest deadline 2019-12-09T19:52:15.930370+0000) 2019-12-09T19:52:19.250 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:19.243+0000 1694f700 -1 osd.4 398 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e396) 2019-12-09T19:52:19.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:52:19.260+0000 1694f700 -1 osd.6 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:01.124768+0000 front 2019-12-09T19:51:01.210688+0000 (oldest deadline 2019-12-09T19:51:46.366284+0000) 2019-12-09T19:52:19.797 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:52:19.785+0000 1694f700 -1 osd.0 398 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:51:06.047469+0000 front 2019-12-09T19:51:05.929766+0000 (oldest deadline 2019-12-09T19:52:04.268725+0000) 2019-12-09T19:52:19.928 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:19.833+0000 36f91700 -1 osd.4 399 build_incremental_map_msg missing incremental map 397 2019-12-09T19:52:19.928 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:52:19.834+0000 36f91700 -1 osd.4 399 build_incremental_map_msg missing incremental map 398 2019-12-09T19:52:39.507 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:31:31.353 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.521 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:31:31.372 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.549 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:31:31.374 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.552 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:31:31.377 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.555 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:31:31.380 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.587 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:31:31.412 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.680 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:31:31.506 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.743 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:31:31.568 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:39.870 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:31:31.720 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:52:59.845 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:52:59.831+0000 32f89700 -1 osd.3 398 build_incremental_map_msg missing incremental map 396 2019-12-09T19:53:03.474 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:53:03.433+0000 32f89700 -1 osd.0 402 build_incremental_map_msg missing incremental map 402 2019-12-09T19:53:03.494 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:53:03.453+0000 31786700 -1 osd.0 402 build_incremental_map_msg missing incremental map 402 2019-12-09T19:53:03.850 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:53:03.751+0000 37f93700 -1 osd.0 402 build_incremental_map_msg missing incremental map 402 2019-12-09T19:53:06.907 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:53:06.793+0000 34f8d700 -1 osd.3 404 build_incremental_map_msg missing incremental map 404 2019-12-09T19:53:08.002 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:53:07.777+0000 32f89700 -1 osd.3 404 build_incremental_map_msg missing incremental map 404 2019-12-09T19:53:08.824 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:53:08.816+0000 33f8b700 -1 osd.3 404 build_incremental_map_msg missing incremental map 404 2019-12-09T19:54:05.321 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:05.317+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:05.858 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:05.854+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:06.346 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:06.342+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:06.426 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:06.422+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:06.887 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:06.883+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:07.240 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:07.235+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:07.397 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:07.393+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:07.411 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:07.407+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:07.919 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:07.913+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:07.973 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:07.968+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:08.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:08.256+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:08.275 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:08.271+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:08.428 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:08.424+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:08.435 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:08.431+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:08.928 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:08.925+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:09.027 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:09.023+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:09.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:09.105+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:09.257 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:09.253+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:09.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:09.256+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:09.458 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:09.454+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:09.480 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:09.473+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:09.913 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:09.909+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:10.077 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:10.073+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:10.143 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:10.139+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:10.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:10.209+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:10.215 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:10.211+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:10.440 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:10.435+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:10.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:10.439+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:10.928 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:10.924+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:11.088 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:11.083+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:11.180 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:11.176+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:11.189 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:11.184+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:11.217 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:11.212+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:11.395 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:11.391+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:11.486 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:11.482+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:11.930 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:11.926+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:12.127 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:12.123+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:12.178 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:12.174+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:12.200 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:12.195+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:12.201 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:12.197+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:12.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:12.417+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:12.477 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:12.473+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:12.929 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:12.924+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:13.124 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:13.120+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:13.182 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:13.178+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:13.183 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:13.179+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:13.211 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:13.206+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:13.457 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:13.453+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:13.466 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:13.461+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:13.964 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:13.959+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:14.154 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:14.150+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:14.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:14.166+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:14.193 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:14.189+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:14.223 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:14.218+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:14.489 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:14.484+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:14.494 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:14.490+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:14.956 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:14.953+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:15.117 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:15.113+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:15.165 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:15.160+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:15.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:15.193+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:15.268 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:15.264+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:15.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:15.455+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:15.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:15.467+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:15.982 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:15.973+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:16.075 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:16.070+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:16.183 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:16.178+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:16.196 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:16.184+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:16.233 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:16.229+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:16.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:16.441+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:16.457 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:16.451+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:17.011 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:17.006+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:17.125 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:17.121+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:17.155 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:17.149+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:17.176 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:17.172+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:17.215 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:17.211+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:17.442 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:17.437+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:17.492 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:17.489+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:18.000 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:17.995+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:18.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:18.137+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:18.169 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:18.165+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:18.223 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:18.219+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:18.242 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:18.238+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:18.428 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:18.425+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:18.455 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:18.450+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:19.025 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:19.020+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:19.131 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:19.126+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:19.150 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:19.146+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:19.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:19.192+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:19.230 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:19.226+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:19.397 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:19.393+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:19.482 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:19.477+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:20.071 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:20.067+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:20.127 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:20.124+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:20.159 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:20.155+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:20.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:20.236+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:20.246 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:20.241+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:20.397 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:20.393+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:20.515 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:20.511+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:21.084 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:21.079+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:21.118 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:21.112+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:21.118 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:21.114+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:21.203 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:21.199+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:21.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:21.293+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:21.393 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:21.386+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:21.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:21.471+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:22.129 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:22.126+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:22.132 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:22.128+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:22.156 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:22.152+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:22.233 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:22.228+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:22.253 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:22.249+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:22.372 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:22.367+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:22.466 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:22.462+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:23.094 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:23.091+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:23.119 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:23.116+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:23.165 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:23.161+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:23.244 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:23.239+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:23.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:23.269+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:23.328 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:23.324+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:23.484 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:23.480+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:24.071 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:24.067+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:24.093 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:24.089+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:24.195 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:24.191+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:24.213 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:24.208+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:24.264 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:24.260+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:24.338 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:24.335+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:24.529 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:24.526+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:25.115 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:25.110+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:25.123 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:25.119+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:25.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:25.197+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:25.251 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:25.247+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:25.252 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:25.246+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:25.352 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:25.347+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:25.508 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:25.501+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:26.122 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:26.119+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:26.154 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:26.150+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:26.170 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:26.166+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:26.224 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:26.217+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:26.275 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:26.265+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:26.398 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:26.392+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:26.535 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:26.531+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:27.093 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:27.087+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:27.126 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:27.122+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:27.181 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:27.176+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:27.223 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:27.220+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:27.321 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:27.315+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:27.442 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:27.437+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:27.586 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:27.581+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:28.072 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:28.068+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:28.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:28.124+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:28.170 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:28.163+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:28.181 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:28.178+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:28.287 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:28.282+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:28.434 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:28.427+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:28.600 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:28.596+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:29.078 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:29.072+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:29.116 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:29.112+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:29.157 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:29.152+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:29.215 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:29.209+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:29.309 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:29.304+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:29.427 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:29.423+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:29.649 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:29.644+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:30.051 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:30.048+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:30.102 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:30.098+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:30.158 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:30.150+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:30.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:30.259+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:30.280 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:30.275+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:30.402 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:30.398+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:30.657 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:30.653+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:31.072 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:31.067+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:31.099 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:31.094+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:31.138 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:31.134+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:31.256 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:31.251+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:31.260 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:31.257+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:31.385 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:31.382+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:31.636 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:31.632+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:32.092 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:32.087+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:32.119 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:32.115+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:32.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:32.128+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:32.294 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:32.291+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:32.303 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:32.299+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:32.352 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:32.349+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:32.609 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:32.604+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:33.089 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:33.084+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:33.162 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:33.157+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:33.178 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:33.173+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:33.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:33.307+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:33.329 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:33.325+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:33.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:33.351+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:33.592 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:33.587+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:34.137 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:34.131+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:34.139 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:34.133+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:34.168 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:34.164+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:34.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:34.290+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:34.345 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:34.340+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:34.400 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:34.395+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:34.589 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:34.584+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:35.126 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:35.121+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:35.170 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:35.164+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:35.180 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:35.176+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:35.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:35.290+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:35.316 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:35.312+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:35.410 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:35.406+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:35.622 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:35.615+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:36.144 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:36.139+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:36.159 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:36.154+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:36.175 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:36.169+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:36.333 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:36.327+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:36.342 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:36.338+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:36.445 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:36.441+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:36.620 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:36.613+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:37.141 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:37.135+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:37.216 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:37.210+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:37.222 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:37.217+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:37.329 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:37.324+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:37.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:37.389+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:37.488 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:37.484+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:37.638 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:37.633+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:38.110 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:38.104+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:38.214 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:38.209+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:38.264 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:38.259+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:38.296 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:38.292+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:38.419 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:38.415+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:38.450 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:38.445+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:38.598 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:38.593+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:39.134 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:39.129+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:39.179 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:39.174+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:39.237 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:39.231+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:39.275 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:39.270+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:39.426 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:39.423+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:39.472 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:39.468+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:39.635 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:39.630+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:39.900 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:54:39.894+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e404 v404) 2019-12-09T19:54:40.135 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:40.129+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:40.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:40.137+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:40.257 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:40.252+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:40.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:40.306+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:40.386 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:40.380+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:40.450 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:40.445+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:40.618 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:40.614+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:41.098 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:41.093+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:41.105 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:41.099+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:41.276 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:41.271+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:41.360 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:41.355+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:41.379 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:41.374+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:41.431 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:41.427+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:41.672 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:41.666+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:42.077 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:42.071+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:42.121 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:42.106+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:42.270 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:42.263+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:42.372 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:42.367+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:42.383 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:42.378+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:42.426 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:42.421+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:42.705 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:42.700+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:43.053 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:43.047+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:43.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:43.067+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:43.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:43.268+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:43.328 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:43.324+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:43.362 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:43.357+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:43.393 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:43.388+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:43.695 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:43.690+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:44.017 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:44.012+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:44.080 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:44.075+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:44.237 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:44.233+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:44.354 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:44.349+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:44.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:44.359+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:44.432 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:44.428+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:44.747 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:44.741+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:44.929 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:54:44.922+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e404 v404) 2019-12-09T19:54:45.053 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:45.043+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:45.123 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:45.118+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:45.268 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:45.264+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:45.376 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:45.371+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:45.385 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:45.380+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:45.411 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:45.407+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:45.748 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:45.743+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:46.084 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:46.080+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:46.095 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:46.091+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:46.230 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:46.224+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:46.359 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:46.355+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:46.363 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:46.359+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:46.461 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:46.457+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:46.763 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:46.758+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:47.066 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:47.061+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:47.149 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:47.144+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:47.234 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:47.229+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:47.377 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:47.373+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:47.384 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:47.380+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:47.420 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:47.416+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:47.762 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:47.757+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:48.074 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:48.070+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:48.149 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:48.142+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:48.223 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:48.217+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:48.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:48.359+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:48.392 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:48.387+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:48.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:48.452+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:48.737 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:48.732+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:49.050 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:49.044+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:49.183 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:49.177+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:49.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:49.179+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:49.322 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:54:49.317+0000 1694f700 -1 osd.7 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.856089+0000 front 2019-12-09T19:53:22.899990+0000 (oldest deadline 2019-12-09T19:54:05.014304+0000) 2019-12-09T19:54:49.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:54:49.388+0000 1694f700 -1 osd.5 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.598882+0000 front 2019-12-09T19:53:23.662720+0000 (oldest deadline 2019-12-09T19:54:08.545462+0000) 2019-12-09T19:54:49.410 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:54:49.406+0000 1694f700 -1 osd.6 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:19.719367+0000 front 2019-12-09T19:53:19.676293+0000 (oldest deadline 2019-12-09T19:54:05.175159+0000) 2019-12-09T19:54:49.699 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:54:49.693+0000 1694f700 -1 osd.3 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:22.936651+0000 front 2019-12-09T19:53:23.162187+0000 (oldest deadline 2019-12-09T19:54:06.246550+0000) 2019-12-09T19:54:50.003 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:54:49.996+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 44sec e404 v404) 2019-12-09T19:54:50.098 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:54:50.081+0000 1694f700 -1 osd.1 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:24.996321+0000 front 2019-12-09T19:53:25.024658+0000 (oldest deadline 2019-12-09T19:54:06.495944+0000) 2019-12-09T19:54:50.160 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:54:50.142+0000 1694f700 -1 osd.2 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:21.842892+0000 front 2019-12-09T19:53:22.176796+0000 (oldest deadline 2019-12-09T19:54:07.076874+0000) 2019-12-09T19:54:50.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:54:50.209+0000 1694f700 -1 osd.0 404 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:53:23.665884+0000 front 2019-12-09T19:53:23.744784+0000 (oldest deadline 2019-12-09T19:54:07.737688+0000) 2019-12-09T19:54:50.247 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:33:42.097 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.261 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:33:42.108 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.270 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:33:42.095 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.273 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:33:42.098 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.273 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:33:42.099 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.278 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:33:42.103 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.409 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:33:42.259 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.523 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:33:42.348 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:54:50.566 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:33:42.391 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:55:48.852 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:48.848+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:49.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:49.034+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:49.273 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:49.268+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:49.289 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:49.284+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:49.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:49.869+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:50.020 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:50.012+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:50.261 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:50.257+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:50.274 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:50.270+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:50.326 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:50.322+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:50.921 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:50.915+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:51.009 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:51.004+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:51.282 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:51.276+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:51.310 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:51.305+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:51.330 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:51.326+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:51.502 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:51.497+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:51.889 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:51.885+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:52.000 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:51.995+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:52.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:52.297+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:52.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:52.336+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:52.368 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:52.363+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:52.519 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:52.513+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:52.849 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:52.845+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:53.006 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:53.001+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:53.332 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:53.328+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:53.372 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:53.367+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:53.395 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:53.390+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:53.526 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:53.521+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:53.885 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:53.879+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:54.016 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:54.012+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:54.375 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:54.370+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:54.381 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:54.377+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:54.388 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:54.384+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:54.551 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:54.547+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:54.866 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:54.861+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:54.987 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:54.983+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:55.389 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:55.385+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:55.427 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:55.423+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:55.429 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:55.424+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:55.579 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:55.574+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:55.882 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:55.876+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:55.998 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:55.993+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:56.417 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:56.411+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:56.439 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:56.434+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:56.445 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:56.441+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:56.598 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:56.591+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:56.868 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:56.864+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:56.955 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:56.949+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:57.381 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:57.377+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:57.407 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:57.401+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:57.438 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:57.433+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:57.598 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:57.594+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:57.900 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:57.897+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:57.941 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:57.937+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:58.338 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:58.333+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:58.370 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:58.365+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:58.447 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:58.443+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:58.631 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:58.627+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:58.906 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:58.902+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:55:58.915 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:58.911+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:59.364 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:55:59.360+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:55:59.367 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:55:59.363+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:55:59.447 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:55:59.441+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:55:59.592 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:55:59.587+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:55:59.885 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:55:59.880+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:55:59.950 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:55:59.945+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:00.323 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:00.318+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:00.367 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:00.361+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:00.449 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:00.440+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:00.636 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:00.631+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:00.885 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:00.880+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:00.994 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:00.990+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:01.279 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:01.274+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:01.417 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:01.411+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:01.449 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:01.434+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:01.676 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:01.671+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:01.896 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:01.889+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:02.000 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:01.994+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:02.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:02.243+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:02.444 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:02.440+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:02.467 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:02.461+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:02.677 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:02.672+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:02.897 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:02.893+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:03.041 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:03.036+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:03.257 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:03.251+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:03.423 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:03.417+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:03.429 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:03.424+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:03.725 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:03.720+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:03.904 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:03.900+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:04.086 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:04.082+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:04.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:04.301+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:04.393 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:04.389+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:04.442 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:04.437+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:04.762 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:04.757+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:04.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:04.884+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:05.075 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:05.068+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:05.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:05.317+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:05.367 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:05.362+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:05.484 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:05.480+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:05.752 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:05.746+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:05.859 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:05.853+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:06.101 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:06.096+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:06.344 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:06.338+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:06.367 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:06.362+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:06.518 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:06.514+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:06.729 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:06.724+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:06.921 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:06.885+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:07.071 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:07.065+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:07.352 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:07.346+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:07.396 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:07.391+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:07.412 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:07.408+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:07.418 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:07.414+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:07.531 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:07.522+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:07.781 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:07.772+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:07.926 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:07.921+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:08.112 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:08.107+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:08.357 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:08.353+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:08.363 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:08.359+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:08.386 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:08.382+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:08.395 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:08.386+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:08.562 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:08.558+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:08.816 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:08.810+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:08.954 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:08.949+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:09.102 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:09.096+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:09.335 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:09.328+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:09.391 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:09.383+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:09.430 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:09.426+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:09.431 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:09.427+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:09.612 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:09.607+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:09.799 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:09.796+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:09.956 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:09.950+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:10.138 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:10.134+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:10.349 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:10.342+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:10.395 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:10.390+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:10.395 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:10.392+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:10.432 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:10.427+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:10.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:10.624+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:10.825 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:10.820+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:10.937 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:10.932+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:11.180 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:11.172+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:11.330 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:11.326+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:11.394 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:11.388+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:11.399 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:11.394+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:11.433 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:11.427+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:11.595 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:11.591+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:11.819 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:11.814+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:11.981 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:11.976+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:12.187 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:12.182+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:12.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:12.296+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:12.373 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:12.367+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:12.376 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:12.369+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:12.437 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:12.432+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:12.582 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:12.578+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:12.843 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:12.838+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:12.982 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:12.977+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:13.241 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:13.237+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:13.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:13.329+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:13.397 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:13.392+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:13.406 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:13.400+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:13.407 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:13.403+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:13.550 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:13.546+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:13.821 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:13.815+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:13.972 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:13.968+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:14.264 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:14.260+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:14.303 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:14.299+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:14.399 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:14.392+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:14.440 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:14.435+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:14.441 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:14.437+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:14.519 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:14.515+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:14.788 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:14.781+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:14.977 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:14.968+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:15.273 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:15.263+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:15.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:15.316+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:15.390 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:15.387+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:15.410 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:15.405+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:15.411 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:15.407+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:15.501 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:15.497+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:15.768 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:15.763+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:15.933 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:15.928+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:16.234 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:16.230+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:16.285 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:16.282+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:16.419 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:16.415+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:16.424 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:16.419+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:16.429 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:16.426+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:16.540 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:16.536+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:16.811 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:16.807+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:16.900 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:16.895+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:17.268 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:17.263+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:17.283 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:17.279+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:17.413 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:17.408+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:17.463 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:17.458+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:17.467 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:17.463+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:17.553 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:17.550+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:17.835 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:17.830+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:17.903 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:17.898+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:18.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:18.221+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:18.250 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:18.244+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:18.415 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:18.410+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:18.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:18.514+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:18.521 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:18.516+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:18.579 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:18.575+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:18.792 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:18.786+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:18.881 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:18.876+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:19.223 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:19.219+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:19.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:19.283+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:19.416 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:19.413+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:19.541 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:19.523+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:19.541 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:19.525+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:19.596 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:19.592+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:19.829 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:19.825+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:19.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:19.884+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:20.237 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:20.231+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:20.263 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:20.259+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:20.295 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:20.288+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:20.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:20.439+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:20.520 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:20.515+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:20.523 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:20.520+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:20.620 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:20.612+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:20.858 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:20.853+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:20.867 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:20.862+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:21.251 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:21.245+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:21.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:21.309+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:21.493 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:21.488+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:21.547 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:21.540+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:21.549 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:21.543+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:21.640 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:21.637+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:21.909 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:21.905+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:21.911 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:21.906+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:22.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:22.217+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:22.337 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:22.330+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:22.500 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:22.495+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:22.551 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:22.545+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:22.557 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:22.548+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:22.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:22.666+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:22.910 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:22.905+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:22.910 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:22.905+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:23.195 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:23.189+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:23.291 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:23.286+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:23.476 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:23.471+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:23.546 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:23.540+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:23.547 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:23.543+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:23.671 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:23.666+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:23.902 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:23.896+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:23.940 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:23.935+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:24.205 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:24.200+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:24.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:24.276+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:24.442 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:24.438+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:24.558 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:24.549+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:24.559 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:24.551+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:24.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:24.642+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:24.897 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:24.893+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:24.920 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:24.914+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:25.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:25.245+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:25.264 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:25.262+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:25.306 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:25.302+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:25.397 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:25.391+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:25.595 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:25.588+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:25.597 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:25.590+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:25.667 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:25.664+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:25.926 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:25.922+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:25.936 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:25.932+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:26.236 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:26.232+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:26.269 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:26.265+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:26.386 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:26.381+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:26.626 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:26.621+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:26.627 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:26.623+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:26.659 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:26.655+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:26.981 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:26.976+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:26.985 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:26.980+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:27.241 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:27.235+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:27.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:27.281+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:27.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:27.341+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:27.650 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:27.644+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:27.651 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:27.647+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:27.685 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:27.680+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:27.954 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:27.949+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:27.984 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:27.981+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:28.249 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:28.243+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:28.286 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:28.281+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:28.313 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:28.309+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:28.625 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:28.619+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:28.626 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:28.621+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:28.696 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:28.693+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:28.929 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:28.925+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:28.987 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:28.982+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:29.236 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:29.232+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:29.279 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:29.275+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:29.321 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:29.317+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:29.634 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:29.628+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:29.635 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:29.631+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:29.747 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:29.744+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:29.901 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:29.896+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:30.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:30.028+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:30.276 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:30.271+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:30.300 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:30.290+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:30.330 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:30.323+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:30.354 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:30.351+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:30.641 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:30.634+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:30.642 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:30.636+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:30.802 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:30.798+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:30.925 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:30.920+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:31.059 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:31.054+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:31.264 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:31.260+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:31.295 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:31.290+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:31.408 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:31.404+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:31.602 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:31.596+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:31.602 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:31.598+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:31.852 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:31.848+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:31.927 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:31.922+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:32.104 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:32.099+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:32.256 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:32.251+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:32.265 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:32.261+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:32.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:32.437+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:32.603 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:32.598+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:32.613 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:32.603+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:32.889 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:32.885+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:32.923 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:32.918+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:33.143 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:33.138+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:33.302 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:33.296+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:33.311 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:33.307+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:33.482 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:33.478+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:33.650 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:33.644+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:33.651 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:33.647+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:33.843 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:33.839+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:33.917 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:33.912+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:34.117 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:34.112+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:34.280 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:34.275+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:34.282 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:34.278+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:34.487 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:34.481+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:34.645 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:34.639+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:34.646 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:34.642+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:34.801 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:34.798+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:34.875 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:34.871+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:35.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:35.129+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:35.247 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:35.243+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:35.274 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:35.270+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:35.352 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:35.344+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:35.491 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:35.487+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:35.637 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:35.632+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:35.638 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:35.634+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:35.813 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:35.810+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:35.844 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:35.839+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:36.147 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:36.142+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:36.268 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:36.263+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:36.320 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:36.316+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:36.525 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:36.522+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:36.654 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:36.649+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:36.654 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:36.651+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:36.815 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:36.811+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:36.820 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:36.816+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:37.198 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:37.192+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:37.223 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:37.217+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:37.320 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:37.316+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:37.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:37.563+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:37.691 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:37.686+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:37.694 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:37.690+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:37.780 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:37.776+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:37.881 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:37.871+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:38.183 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:38.176+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:38.276 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:38.268+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:38.355 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:38.351+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:38.608 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:38.605+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:38.722 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:38.716+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:38.723 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:38.718+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:38.739 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:38.734+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:38.879 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:38.874+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:39.189 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:39.184+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:39.238 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:39.233+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:39.376 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:39.373+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:39.570 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:39.567+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:39.689 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:39.684+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:39.693 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:39.689+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:39.695 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:39.691+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:39.865 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:39.860+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:40.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:40.145+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:40.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:40.234+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:40.375 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:40.365+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:40.399 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:40.393+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:40.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:40.614+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:40.710 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:40.705+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:40.718 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:40.711+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:40.718 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:40.707+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:40.853 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:40.849+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:41.148 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:41.145+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:41.270 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:41.266+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:41.453 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:41.448+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:41.633 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:41.628+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:41.728 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:41.723+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:41.743 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:41.740+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:41.744 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:41.741+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:41.896 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:41.894+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:42.207 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:42.203+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:42.257 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:42.253+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:42.506 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:42.500+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:42.644 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:42.634+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:42.753 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:42.748+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:42.760 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:42.757+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:42.772 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:42.769+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:42.936 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:42.931+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:43.236 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:43.231+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:43.250 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:43.245+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:43.554 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:43.550+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:43.593 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:43.589+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:43.732 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:43.726+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:43.736 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:43.728+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:43.741 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:43.738+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:43.904 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:43.900+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:44.268 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:44.260+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:44.275 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:44.269+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:44.559 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:44.554+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:44.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:56:44.575+0000 1694f700 -1 osd.2 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.307360+0000 front 2019-12-09T19:55:07.300774+0000 (oldest deadline 2019-12-09T19:55:48.986049+0000) 2019-12-09T19:56:44.699 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:44.696+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:44.727 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:44.721+0000 1694f700 -1 osd.4 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:55:27.117118+0000 (oldest deadline 2019-12-09T19:56:07.117118+0000) 2019-12-09T19:56:44.728 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:44.723+0000 1694f700 -1 osd.4 410 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:44.936 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:44.931+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:56:45.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:56:45.260+0000 1694f700 -1 osd.6 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.476412+0000 front 2019-12-09T19:55:07.543199+0000 (oldest deadline 2019-12-09T19:55:50.950952+0000) 2019-12-09T19:56:45.271 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:56:45.267+0000 1694f700 -1 osd.7 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.565128+0000 front 2019-12-09T19:55:04.535834+0000 (oldest deadline 2019-12-09T19:55:48.020163+0000) 2019-12-09T19:56:45.455 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:56:45.446+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e410 v410) 2019-12-09T19:56:45.532 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:56:45.528+0000 1694f700 -1 osd.1 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:04.425008+0000 front 2019-12-09T19:55:04.529774+0000 (oldest deadline 2019-12-09T19:55:49.676599+0000) 2019-12-09T19:56:45.662 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:56:45.658+0000 1694f700 -1 osd.0 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:05.862786+0000 front 2019-12-09T19:55:06.280515+0000 (oldest deadline 2019-12-09T19:55:48.449015+0000) 2019-12-09T19:56:45.802 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:56:45.744+0000 1694f700 -1 osd.4 411 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4385.0:120009 7.4 7.8ff4d744 (undecoded) ondisk+retry+write+known_if_redirected e408) 2019-12-09T19:56:45.918 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:56:45.915+0000 1694f700 -1 osd.5 410 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:55:07.543314+0000 front 2019-12-09T19:55:07.571165+0000 (oldest deadline 2019-12-09T19:55:48.607702+0000) 2019-12-09T19:57:12.652 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:12.623+0000 31f87700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:12.783 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:12.747+0000 34f8d700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:12.928 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:12.910+0000 11caf700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:12.986 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:12.965+0000 32788700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:13.008 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:12.987+0000 36f91700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:13.181 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:13.065+0000 3378a700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:13.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:13.342+0000 38794700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:57:14.069 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:57:14.041+0000 37f93700 -1 osd.3 416 build_incremental_map_msg missing incremental map 416 2019-12-09T19:58:10.330 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:10.324+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:11.311 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:11.306+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:12.302 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:12.297+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:12.427 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:12.421+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:13.303 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:13.299+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:13.407 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:13.401+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:13.771 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:13.767+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:14.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:14.297+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:14.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:14.360+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:14.634 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:14.630+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:14.740 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:14.737+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:15.195 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:15.190+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:15.334 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:15.329+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:15.352 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:15.348+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:15.627 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:15.620+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:15.713 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:15.699+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:16.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:16.201+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:16.313 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:16.308+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:16.386 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:16.382+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:16.583 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:16.579+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:16.583 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:16.579+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:16.697 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:16.693+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:17.179 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:17.175+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:17.357 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:17.345+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:17.377 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:17.371+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:17.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:17.543+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:17.636 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:17.632+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:17.742 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:17.738+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:18.175 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:18.171+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:18.371 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:18.366+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:18.379 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:18.375+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:18.510 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:18.506+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:18.593 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:18.588+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:18.775 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:18.771+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:19.148 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:19.144+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:19.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:19.338+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:19.420 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:19.415+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:19.551 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:19.546+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:19.554 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:19.549+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:19.752 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:19.748+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:20.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:20.151+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:20.312 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:20.307+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:20.453 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:20.448+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:20.525 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:20.518+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:20.542 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:20.537+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:20.718 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:20.712+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:21.184 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:21.179+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:21.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:21.278+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:21.476 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:21.470+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:21.481 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:21.476+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:21.545 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:21.540+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:21.717 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:21.712+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:22.171 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:22.166+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:22.239 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:22.235+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:22.504 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:22.499+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:22.516 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:22.505+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:22.539 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:22.533+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:22.698 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:22.693+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:23.200 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:23.195+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:23.202 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:23.198+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:23.495 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:23.490+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:23.510 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:23.504+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:23.590 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:23.585+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:23.728 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:23.722+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:24.171 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:24.167+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:24.231 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:24.226+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:24.494 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:24.489+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:24.560 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:24.556+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:24.564 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:24.559+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:24.776 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:24.772+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:25.211 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:25.206+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:25.283 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:25.278+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:25.527 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:25.522+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:25.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:25.529+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:25.553 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:25.549+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:25.756 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:25.750+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:26.240 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:26.235+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:26.331 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:26.326+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:26.544 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:26.539+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:26.579 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:26.573+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:26.598 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:26.593+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:26.784 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:26.779+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:27.270 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:27.265+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:27.339 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:27.334+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:27.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:27.560+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:27.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:27.574+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:27.628 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:27.623+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:27.810 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:27.806+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:28.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:28.232+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:28.294 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:28.290+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:28.559 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:28.553+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:28.576 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:28.571+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:28.606 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:28.599+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:28.777 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:28.772+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:29.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:29.236+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:29.314 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:29.309+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:29.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:29.530+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:29.546 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:29.541+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:29.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:29.571+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:29.816 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:29.811+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:30.202 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:30.198+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:30.334 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:30.329+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:30.537 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:30.533+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:30.576 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:30.572+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:30.600 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:30.596+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:30.788 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:30.783+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:31.242 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:31.237+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:31.330 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:31.325+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:31.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:31.531+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:31.539 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:31.535+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:31.626 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:31.620+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:31.765 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:31.761+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:32.199 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:32.195+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:32.356 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:32.352+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:32.519 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:32.514+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:32.547 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:32.541+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:32.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:32.592+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:32.738 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:32.725+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:33.184 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:33.180+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:33.392 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:33.388+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:33.530 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:33.520+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:33.590 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:33.584+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:33.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:33.624+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:33.691 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:33.687+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:34.181 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:34.171+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:34.391 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:34.386+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:34.551 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:34.545+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:34.636 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:34.630+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:34.646 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:34.642+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:34.669 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:34.659+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:35.154 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:35.150+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:35.389 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:35.385+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:35.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:35.542+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:35.620 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:35.615+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:35.656 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:35.652+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:35.657 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:35.653+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:36.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:36.159+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:36.436 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:36.431+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:36.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:36.555+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:36.651 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:36.647+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:36.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:36.663+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:36.671 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:36.666+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:37.170 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:37.163+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:37.432 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:37.428+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:37.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:37.569+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:37.647 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:37.641+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:37.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:37.667+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:37.696 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:37.690+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:38.159 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:38.155+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:38.428 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:38.423+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:38.613 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:38.608+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:38.666 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:38.661+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:38.689 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:38.685+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:38.698 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:38.692+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:39.164 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:39.159+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:39.478 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:39.474+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:39.584 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:39.579+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:39.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:39.645+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:39.961 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:39.723+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:39.961 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:39.728+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:40.168 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:40.163+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:40.521 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:40.517+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:40.584 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:40.580+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:40.664 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:40.659+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:40.739 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:40.735+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:40.774 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:40.769+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:40.853 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:58:40.848+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e416 v416) 2019-12-09T19:58:41.139 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:41.134+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:41.555 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:41.549+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:41.607 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:41.601+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:41.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:41.635+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:41.730 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:41.724+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:41.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:41.767+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:42.180 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:42.175+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:42.512 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:42.507+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:42.610 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:42.604+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:42.613 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:42.601+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:42.723 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:42.717+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:42.818 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:42.813+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:43.175 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:43.170+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:43.517 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:43.511+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:43.599 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:43.595+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:43.637 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:43.633+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:43.710 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:43.707+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:43.795 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:43.790+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:44.214 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:44.208+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:44.548 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:44.544+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:44.608 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:44.603+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:44.631 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:44.627+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:44.763 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:44.758+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:44.763 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:44.758+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:45.194 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:45.189+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:45.542 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:45.538+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:45.581 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:45.573+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:45.673 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:45.669+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:45.776 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:45.772+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:45.789 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:45.785+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:45.879 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:58:45.875+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e416 v416) 2019-12-09T19:58:46.166 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:46.161+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:46.521 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:46.516+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:46.602 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:46.598+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:46.682 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:46.678+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:46.790 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:46.785+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:46.816 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:46.812+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:47.151 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:47.147+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:47.550 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:47.545+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:47.642 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:47.639+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:47.689 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:47.684+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:47.782 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:47.779+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:47.790 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:47.786+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:48.116 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:37:39.967 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.123 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:37:39.973 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.132 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:37:39.954 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.132 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:37:39.957 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.134 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:37:39.960 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.141 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:37:39.966 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.160 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:48.155+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:48.353 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:37:40.203 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.375 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:37:40.200 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.380 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:37:40.205 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T19:58:48.598 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:48.594+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:48.616 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:48.613+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:48.660 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:48.657+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:48.809 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:48.806+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:48.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:48.830+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:49.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:49.013+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:49.167 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:49.163+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:49.598 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:49.595+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:49.640 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:49.637+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:49.648 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:49.645+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:49.856 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:49.853+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:49.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:49.858+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:49.974 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:49.972+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:50.167 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:50.163+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:50.611 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:50.609+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:50.673 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:50.670+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:50.691 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:50.677+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:50.873 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:50.868+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:50.893 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:58:50.890+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e416 v416) 2019-12-09T19:58:50.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:50.911+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:50.935 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:50.932+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:51.209 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:51.206+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:51.632 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:51.629+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:51.702 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:51.697+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:51.722 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:51.718+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:51.893 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:51.890+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:51.926 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:51.921+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:51.957 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:51.952+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:52.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:52.218+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:52.590 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:52.584+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:52.741 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:52.737+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:52.769 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:52.765+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:52.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:52.878+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:52.973 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:52.969+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:52.984 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:52.976+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:53.212 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:53.208+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:53.562 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:53.557+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:53.744 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:53.740+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:53.793 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:53.789+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:53.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:53.843+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:53.954 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:53.948+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:54.007 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:54.003+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:54.176 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:54.171+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:54.550 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:54.546+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:54.719 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:54.715+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:54.805 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:54.801+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:54.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:54.896+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:54.990 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:54.984+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:54.998 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:54.993+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:55.221 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:58:55.216+0000 1694f700 -1 osd.6 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.558762+0000 front 2019-12-09T19:57:30.540289+0000 (oldest deadline 2019-12-09T19:58:14.426695+0000) 2019-12-09T19:58:55.545 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T19:58:55.540+0000 1694f700 -1 osd.5 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.920944+0000 front 2019-12-09T19:57:30.932945+0000 (oldest deadline 2019-12-09T19:58:12.279604+0000) 2019-12-09T19:58:55.715 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:58:55.710+0000 1694f700 -1 osd.1 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.977213+0000 front 2019-12-09T19:57:29.868759+0000 (oldest deadline 2019-12-09T19:58:14.095766+0000) 2019-12-09T19:58:55.813 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:58:55.809+0000 1694f700 -1 osd.0 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:28.257350+0000 front 2019-12-09T19:57:28.586504+0000 (oldest deadline 2019-12-09T19:58:09.967489+0000) 2019-12-09T19:58:55.942 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:58:55.937+0000 1694f700 -1 osd.2 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:30.569153+0000 front 2019-12-09T19:57:30.567217+0000 (oldest deadline 2019-12-09T19:58:15.754224+0000) 2019-12-09T19:58:55.958 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:58:55.954+0000 1694f700 -1 osd.7 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:31.652382+0000 front 2019-12-09T19:57:31.358879+0000 (oldest deadline 2019-12-09T19:58:48.022729+0000) 2019-12-09T19:58:55.969 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:58:55.965+0000 1694f700 -1 osd.3 416 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T19:57:29.752394+0000 front 2019-12-09T19:57:28.966244+0000 (oldest deadline 2019-12-09T19:58:12.877607+0000) 2019-12-09T19:58:55.973 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T19:58:55.961+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 42sec e416 v416) 2019-12-09T19:59:40.105 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:59:39.985+0000 35f8f700 -1 osd.3 420 build_incremental_map_msg missing incremental map 420 2019-12-09T19:59:40.669 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T19:59:40.391+0000 3378a700 -1 osd.3 420 build_incremental_map_msg missing incremental map 420 2019-12-09T19:59:42.096 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.087+0000 30f85700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.157 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.128+0000 30f85700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.265 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.233+0000 32788700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.288 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.244+0000 35f8f700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.483 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.476+0000 33f8b700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.699 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.617+0000 33f8b700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.714 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.641+0000 38794700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:42.856 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:42.849+0000 3478c700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:43.192 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T19:59:43.128+0000 36790700 -1 osd.4 422 build_incremental_map_msg missing incremental map 422 2019-12-09T19:59:54.323 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:54.318+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:55.059 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:59:55.054+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T19:59:55.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:55.298+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:55.426 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:59:55.420+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T19:59:55.523 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:59:55.519+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T19:59:56.102 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:59:56.096+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T19:59:56.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:56.292+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:56.403 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:59:56.397+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T19:59:56.571 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:59:56.566+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T19:59:57.138 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:59:57.133+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T19:59:57.202 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:59:57.197+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T19:59:57.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:57.291+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:57.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:59:57.423+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T19:59:57.528 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:59:57.523+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T19:59:58.163 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:59:58.158+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T19:59:58.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:59:58.179+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T19:59:58.293 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:58.287+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:58.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:59:58.477+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T19:59:58.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:59:58.529+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T19:59:59.167 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T19:59:59.161+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T19:59:59.225 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T19:59:59.221+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T19:59:59.342 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T19:59:59.338+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T19:59:59.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T19:59:59.452+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T19:59:59.584 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T19:59:59.579+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:00.119 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:00.114+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:00.197 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:00.194+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:00.211 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:00.206+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:00.375 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:00.371+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:00.458 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:00.453+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:00.590 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:00.586+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:01.150 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:01.146+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:01.172 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:01.167+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:01.191 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:01.187+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:01.369 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:01.365+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:01.443 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:01.432+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:01.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:01.556+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:02.139 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:02.133+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:02.151 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:02.146+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:02.198 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:02.194+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:02.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:02.357+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:02.409 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:02.403+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:02.548 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:02.543+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:03.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:03.147+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:03.172 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:03.168+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:03.253 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:03.247+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:03.371 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:03.368+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:03.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:03.423+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:03.556 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:03.553+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:04.190 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:04.185+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:04.212 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:04.200+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:04.226 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:04.222+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:04.373 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:04.370+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:04.383 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:04.380+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:04.575 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:04.571+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:05.184 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:05.180+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:05.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:05.192+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:05.223 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:05.218+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:05.353 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:05.349+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:05.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:05.352+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:05.565 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:05.561+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:06.147 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:06.144+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:06.233 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:06.228+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:06.246 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:06.242+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:06.367 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:06.363+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:06.370 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:06.366+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:06.598 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:06.592+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:07.170 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:07.166+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:07.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:07.220+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:07.233 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:07.228+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:07.343 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:07.339+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:07.350 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:07.344+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:07.578 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:07.574+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:08.199 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:08.193+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:08.200 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:08.196+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:08.251 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:08.246+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:08.301 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:08.297+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:08.393 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:08.388+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:08.608 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:08.597+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:09.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:09.156+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:09.221 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:09.216+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:09.263 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:09.258+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:09.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:09.319+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:09.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:09.390+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:09.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:09.592+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:10.128 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:10.124+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:10.252 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:10.247+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:10.267 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:10.262+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:10.294 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:10.289+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:10.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:10.353+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:10.563 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:10.560+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:10.886 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:10.881+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:11.126 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:11.123+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:11.227 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:11.223+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:11.250 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:11.244+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:11.298 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:11.293+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:11.321 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:11.316+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:11.588 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:11.583+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:11.926 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:11.922+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:12.157 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:12.149+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:12.209 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:12.204+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:12.260 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:12.255+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:12.265 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:12.260+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:12.303 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:12.298+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:12.593 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:12.589+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:12.908 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:12.904+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:13.162 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:13.158+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:13.243 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:13.238+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:13.253 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:13.248+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:13.260 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:13.255+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:13.268 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:13.263+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:13.570 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:13.566+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:13.898 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:13.892+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:14.140 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:14.136+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:14.226 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:14.221+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:14.249 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:14.244+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:14.297 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:14.292+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:14.319 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:14.315+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:14.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:14.543+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:14.854 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:14.850+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:15.128 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:15.124+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:15.191 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:15.187+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:15.234 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:15.230+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:15.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:15.307+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:15.337 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:15.333+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:15.567 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:15.563+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:15.865 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:15.861+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:16.148 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:16.145+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:16.163 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:16.159+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:16.193 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:16.190+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:16.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:16.301+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:16.355 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:16.352+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:16.564 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:16.560+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:16.906 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:16.903+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:17.133 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:17.128+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:17.164 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:17.159+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:17.205 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:17.201+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:17.337 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:17.333+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:17.351 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:17.346+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:17.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:17.570+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:17.896 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:17.889+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:17.897 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:17.892+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:18.142 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:18.138+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:18.177 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:18.168+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:18.177 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:18.169+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:18.350 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:18.345+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:18.370 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:18.365+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:18.532 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:18.527+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:18.939 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:18.932+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:18.940 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:18.936+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:19.118 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:19.114+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:19.151 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:19.147+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:19.160 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:19.154+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:19.351 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:19.346+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:19.359 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:19.354+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:19.548 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:19.542+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:19.949 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:19.944+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:19.950 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:19.946+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:20.114 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:20.110+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:20.151 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:20.148+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:20.157 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:20.154+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:20.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:20.316+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:20.373 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:20.368+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:20.545 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:20.540+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:20.952 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:20.946+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:20.953 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:20.948+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:21.077 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:21.072+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:21.123 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:21.118+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:21.205 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:21.201+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:21.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:21.349+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:21.388 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:21.371+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:21.565 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:21.560+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:21.918 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:21.914+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:21.921 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:21.918+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:22.115 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:22.109+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:22.137 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:22.133+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:22.199 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:22.194+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:22.341 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:22.337+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:22.344 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:22.341+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:22.606 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:22.603+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:22.933 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:22.925+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:22.934 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:22.927+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:23.105 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:23.102+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:23.150 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:23.145+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:23.163 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:23.159+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:23.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:23.306+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:23.347 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:23.342+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:23.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:23.592+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:23.980 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:23.972+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:23.983 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:23.977+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:24.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:24.056+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:24.118 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:24.113+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:24.173 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:24.169+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:24.308 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:24.302+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:24.394 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:24.389+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:24.637 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:24.633+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:24.951 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:24.946+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:24.955 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:24.951+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:25.052 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:25.049+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:25.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:25.148+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:25.190 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:25.186+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:25.346 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:25.340+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:25.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:25.350+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:25.686 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:25.680+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:25.924 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:25.918+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:25.925 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:25.921+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:26.066 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:26.062+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:26.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:26.156+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:26.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:26.224+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:26.253 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:26.250+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:26.316 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:26.310+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:26.335 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:26.330+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:26.719 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:26.715+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:26.896 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:26.890+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:26.897 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:26.893+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:27.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:27.087+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:27.142 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:27.137+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:27.271 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:27.268+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:27.322 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:27.316+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:27.367 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:27.363+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:27.729 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:27.726+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:27.857 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:27.852+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:27.860 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:27.857+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:28.057 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:28.053+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:28.174 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:28.169+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:28.271 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:28.267+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:28.283 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:28.278+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:28.390 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:28.385+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:28.727 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:28.723+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:28.864 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:28.859+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:28.869 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:28.864+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:29.034 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:29.029+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:29.205 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:29.199+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:29.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:29.245+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:29.316 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:29.310+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:29.414 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:29.409+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:29.774 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:29.769+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:29.922 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:29.911+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:29.925 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:29.919+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:30.082 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:30.069+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:30.195 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:30.190+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:30.283 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:30.279+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:30.354 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:30.349+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:30.400 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:30.395+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:30.798 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:30.793+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:30.967 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:30.963+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:30.970 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:30.967+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:31.077 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:31.072+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:31.232 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:31.227+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:31.271 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:31.267+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:31.319 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:31.315+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:31.396 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:31.389+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:31.396 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:31.391+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:31.779 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:31.775+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:31.939 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:31.933+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:31.940 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:31.935+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:32.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:32.110+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:32.258 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:32.254+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:32.310 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:32.304+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:32.370 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:32.365+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:32.388 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:32.384+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:32.778 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:32.774+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:32.914 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:32.908+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:32.918 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:32.910+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:33.112 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:33.109+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:33.235 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:33.228+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:33.297 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:33.294+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:33.340 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:33.334+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:33.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:33.384+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:33.792 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:33.787+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:33.879 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:33.873+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:33.880 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:33.875+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:34.124 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:34.120+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:34.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:34.261+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:34.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:34.310+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:34.378 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:34.373+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:34.445 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:34.439+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:34.838 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:34.832+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:34.871 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:34.866+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:34.875 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:34.871+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:35.119 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:35.115+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:35.234 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:35.229+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:35.325 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:35.314+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:35.412 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:35.407+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:35.455 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:35.449+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:35.804 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:35.799+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:35.830 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:35.824+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:35.830 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:35.827+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:36.171 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:36.168+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:36.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:36.262+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:36.290 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:36.284+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:36.345 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:36.342+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:36.430 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:36.424+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:36.431 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:36.425+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:36.776 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:36.772+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:36.855 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:36.849+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:36.858 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:36.854+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:37.189 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:37.184+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:37.250 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:37.245+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:37.304 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:37.299+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:37.426 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:37.421+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:37.480 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:37.471+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:37.827 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:37.823+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:37.836 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:37.830+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:37.837 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:37.833+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:38.149 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:38.145+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:38.216 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:38.212+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:38.310 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:38.307+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:38.384 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:38.379+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:38.493 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:38.487+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:38.794 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:38.788+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:38.798 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:38.793+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:38.848 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:38.844+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:39.108 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:39.103+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:39.205 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:39.194+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:39.351 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:39.344+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:39.427 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:39.417+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:39.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:39.467+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:39.847 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:39.841+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:39.848 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:39.844+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:39.879 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:39.874+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:40.103 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:40.100+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:40.245 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:40.240+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:40.351 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:40.347+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:40.430 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:40.424+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:40.521 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:40.516+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:40.875 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:40.872+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:40.896 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:40.891+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:40.900 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:40.896+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:41.151 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:41.147+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:41.250 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:41.245+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:41.308 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:41.304+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:41.309 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:41.305+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:41.395 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:41.390+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:41.487 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:41.482+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:41.859 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:41.853+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:41.861 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:41.856+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:41.891 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:41.887+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:42.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:42.157+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:42.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:42.255+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:42.355 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:42.350+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:42.375 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:42.370+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:42.507 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:42.503+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:42.852 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:42.846+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:42.859 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:42.849+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:42.929 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:42.925+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:43.125 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:43.122+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:43.237 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:43.233+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:43.354 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:43.351+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:43.410 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:43.405+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:43.532 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:43.526+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:43.838 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:43.834+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:43.843 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:43.839+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:43.966 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:43.963+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:44.170 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:44.167+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:44.215 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:44.211+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:44.325 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:44.320+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:44.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:44.449+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:44.492 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:44.486+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:44.850 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:44.844+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:44.855 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:44.846+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:44.979 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:44.975+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:45.205 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:45.201+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:45.224 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:45.219+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:45.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:45.329+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:45.436 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:45.431+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:45.501 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:45.496+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:45.900 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:45.894+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:45.901 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:45.896+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:46.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:46.011+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:46.237 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:46.234+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:46.261 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:46.256+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:46.329 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:46.323+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:46.346 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:46.334+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:46.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:46.460+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:46.549 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:46.543+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:46.936 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:46.927+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:46.937 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:46.929+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:47.032 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:47.028+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:47.251 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:47.248+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:47.255 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:47.250+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:47.389 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:47.386+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:47.511 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:47.506+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:47.596 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:47.590+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:47.953 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:47.947+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:47.957 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:47.953+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:48.021 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:48.017+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:48.238 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:48.235+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:48.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:48.237+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:48.356 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:48.352+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:48.554 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:48.549+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:48.624 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:48.618+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:48.974 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:48.961+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:48.974 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:48.964+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:49.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:49.020+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:49.279 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:49.273+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:49.286 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:49.282+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:49.400 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:49.395+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:49.551 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:49.546+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:49.620 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:49.615+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:49.990 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:49.983+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:49.993 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:49.986+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:50.042 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:50.037+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:50.306 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:50.302+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:50.317 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:50.310+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:50.375 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:50.368+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:50.537 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:50.531+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:50.659 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:50.653+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:51.010 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:51.005+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:51.011 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:51.007+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:51.036 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:51.031+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:51.332 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:51.328+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:51.343 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:51.340+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:51.347 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:51.343+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:51.355 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:51.349+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:51.564 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:51.559+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:51.678 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:51.672+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:52.030 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:52.023+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:52.031 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:52.026+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:52.035 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:52.029+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:52.326 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:52.322+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:52.332 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:52.327+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:52.395 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:52.390+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:52.569 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:52.565+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:52.640 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:52.635+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:53.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:53.025+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:53.057 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:53.050+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:53.061 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:53.053+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:53.328 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:53.323+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:53.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:53.336+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:53.381 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:53.376+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:53.582 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:53.577+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:53.657 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:53.652+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:54.076 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:54.070+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:54.088 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:54.082+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:54.091 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:54.085+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:54.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:54.310+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:54.336 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:54.331+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:54.371 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:54.364+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:54.544 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:54.539+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:54.680 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:54.675+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:55.054 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:55.047+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:55.055 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:55.051+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:55.101 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:55.096+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:55.320 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:55.315+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:55.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:55.318+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:55.387 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:55.382+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:55.531 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:55.526+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:55.721 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:00:55.716+0000 1694f700 -1 osd.7 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.783100+0000 front 2019-12-09T19:59:13.996788+0000 (oldest deadline 2019-12-09T19:59:54.774765+0000) 2019-12-09T20:00:56.015 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:56.009+0000 1694f700 -1 osd.4 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 ever on either front or back, first ping sent 2019-12-09T19:59:36.950272+0000 (oldest deadline 2019-12-09T20:00:16.950272+0000) 2019-12-09T20:00:56.016 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:56.011+0000 1694f700 -1 osd.4 422 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4458.0:119474 7.4 7.871b6684 (undecoded) ondisk+retry+read+known_if_redirected e420) 2019-12-09T20:00:56.133 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:00:56.128+0000 1694f700 -1 osd.1 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.548921+0000 front 2019-12-09T19:59:13.749360+0000 (oldest deadline 2019-12-09T19:59:54.554233+0000) 2019-12-09T20:00:56.307 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:00:56.302+0000 1694f700 -1 osd.2 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:10.529219+0000 front 2019-12-09T19:59:13.887970+0000 (oldest deadline 2019-12-09T19:59:53.871527+0000) 2019-12-09T20:00:56.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:00:56.340+0000 1694f700 -1 osd.0 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:11.198353+0000 front 2019-12-09T19:59:11.689560+0000 (oldest deadline 2019-12-09T19:59:56.468735+0000) 2019-12-09T20:00:56.436 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:00:56.426+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 41sec e422 v422) 2019-12-09T20:00:56.438 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:00:56.433+0000 1694f700 -1 osd.5 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.741772+0000 front 2019-12-09T19:59:13.569372+0000 (oldest deadline 2019-12-09T19:59:59.137472+0000) 2019-12-09T20:00:56.571 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:00:56.566+0000 1694f700 -1 osd.6 422 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T19:59:13.985235+0000 front 2019-12-09T19:59:13.985907+0000 (oldest deadline 2019-12-09T19:59:54.737487+0000) 2019-12-09T20:00:57.099 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:00:57.020+0000 36f91700 -1 osd.4 423 build_incremental_map_msg missing incremental map 422 2019-12-09T20:01:01.623 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:39:53.473 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.668 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:39:53.518 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.669 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:39:53.494 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.677 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:39:53.502 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.717 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:39:53.542 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.720 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:39:53.545 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.794 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:39:53.644 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:01.842 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:39:53.667 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:02.051 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:39:53.695 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:16.929 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:40:08.779 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:16.962 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:40:08.812 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:16.997 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:40:08.822 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.011 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:40:08.836 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.029 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:40:08.880 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.039 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:40:08.865 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.047 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:40:08.872 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.101 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:40:08.923 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:01:17.102 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:40:08.923 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:12.570 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:12.564+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:13.223 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:13.217+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:13.619 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:13.614+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:13.648 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:13.642+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:14.255 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:14.250+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:14.642 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:14.632+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:14.645 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:14.640+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:15.105 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:15.100+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:15.258 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:15.252+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:15.596 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:15.590+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:15.665 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:15.657+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:16.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:16.076+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:16.233 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:16.228+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:16.536 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:16.532+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:16.598 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:16.593+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:16.680 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:16.675+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:16.868 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:16.864+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:17.100 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:17.096+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:17.210 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:17.204+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:17.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:17.568+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:17.626 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:17.621+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:17.715 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:17.710+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:17.885 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:17.880+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:17.891 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:17.887+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:18.124 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:18.120+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:18.217 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:18.212+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:18.618 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:18.613+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:18.675 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:18.670+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:18.678 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:18.671+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:18.913 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:18.909+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:18.936 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:18.932+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:19.129 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:19.125+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:19.271 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:19.265+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:19.640 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:19.634+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:19.649 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:19.644+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:19.703 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:19.698+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:19.887 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:19.883+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:19.904 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:19.900+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:20.165 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:20.160+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:20.286 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:20.281+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:20.603 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:20.599+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:20.687 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:20.682+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:20.699 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:20.695+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:20.919 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:20.915+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:20.919 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:20.915+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:21.145 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:21.140+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:21.275 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:21.270+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:21.611 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:21.606+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:21.642 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:21.637+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:21.687 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:21.682+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:21.908 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:21.904+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:21.931 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:21.928+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:22.152 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:22.148+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:22.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:22.303+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:22.572 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:22.566+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:22.656 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:22.650+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:22.667 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:22.662+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:22.868 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:22.865+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:22.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:22.886+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:23.168 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:23.164+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:23.338 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:23.332+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:23.540 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:23.534+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:23.653 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:23.648+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:23.698 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:23.694+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:23.851 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:23.846+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:23.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:23.900+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:24.167 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:24.163+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:24.393 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:24.387+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:24.516 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:24.509+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:24.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:24.640+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:24.722 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:24.718+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:24.823 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:24.821+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:24.941 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:24.938+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:25.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:25.147+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:25.373 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:25.367+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:25.534 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:25.528+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:25.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:25.636+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:25.726 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:25.721+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:25.819 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:25.816+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:25.899 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:25.896+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:26.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:26.196+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:26.333 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:26.328+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:26.586 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:26.581+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:26.636 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:26.630+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:26.773 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:26.768+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:26.871 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:26.867+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:26.926 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:26.921+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:27.223 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:27.218+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:27.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:27.305+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:27.603 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:27.598+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:27.610 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:27.605+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:27.733 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:27.728+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:27.886 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:27.881+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:27.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:27.928+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:28.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:28.175+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:28.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:28.309+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:28.579 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:28.574+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:28.620 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:28.614+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:28.730 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:28.725+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:28.854 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:28.849+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:28.985 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:28.980+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:29.215 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:29.210+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:29.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:29.313+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:29.604 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:29.597+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:29.614 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:29.608+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:29.716 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:29.710+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:29.873 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:29.868+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:29.993 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:29.989+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:30.215 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:30.211+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:30.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:30.296+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:30.619 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:30.615+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:30.624 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:30.620+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:30.702 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:30.698+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:30.883 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:30.877+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:30.952 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:30.947+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:31.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:31.177+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:31.280 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:31.275+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:31.669 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:31.665+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:31.676 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:31.671+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:31.696 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:31.692+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:31.889 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:31.884+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:31.966 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:31.961+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:32.193 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:32.188+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:32.307 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:32.303+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:32.640 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:32.635+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:32.686 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:32.682+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:32.714 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:32.711+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:32.928 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:32.923+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:32.940 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:32.934+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:33.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:33.148+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:33.277 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:33.272+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:33.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:33.593+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:33.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:33.700+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:33.719 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:33.714+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:33.983 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:33.978+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:33.983 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:33.979+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:34.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:34.193+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:34.278 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:34.275+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:34.603 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:34.599+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:34.719 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:34.716+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:34.726 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:34.722+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:34.978 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:34.974+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:35.026 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:35.022+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:35.180 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:35.175+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:35.243 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:35.239+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:35.595 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:35.590+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:35.695 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:35.691+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:35.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:35.707+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:35.949 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:35.944+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:35.999 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:35.995+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:36.194 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:36.190+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:36.244 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:36.241+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:36.582 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:36.578+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:36.698 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:36.695+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:36.756 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:36.753+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:36.988 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:36.983+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:37.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:37.013+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:37.186 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:37.182+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:37.245 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:37.239+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:37.596 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:37.592+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:37.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:37.668+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:37.745 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:37.740+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:38.001 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:37.997+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:38.017 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:38.010+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:38.192 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:38.187+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:38.276 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:38.271+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:38.566 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:38.562+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:38.673 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:38.667+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:38.740 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:38.736+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:39.031 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:39.027+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:39.053 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:39.048+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:39.234 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:39.230+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:39.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:39.236+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:39.593 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:39.588+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:39.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:39.699+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:39.750 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:39.745+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:40.049 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:40.045+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:40.082 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:40.078+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:40.249 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:40.244+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:40.286 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:40.280+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:40.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:40.641+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:40.663 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:40.658+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:40.733 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:40.728+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:41.089 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:41.084+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:41.131 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:41.128+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:41.251 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:41.248+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:41.322 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:41.318+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:41.608 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:41.604+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:41.688 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:41.684+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:41.754 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:41.749+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:42.138 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:42.133+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:42.186 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:42.182+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:42.246 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:42.242+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:42.286 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:41:34.136 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.297 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:41:34.148 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.308 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:41:34.133 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.309 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:41:34.134 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.313 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:41:34.138 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.317 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:41:34.142 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:42.343+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:42.431 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:41:34.281 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.441 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:41:34.266 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.449 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:41:34.273 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:02:42.574 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:42.571+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:42.701 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:42.696+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:42.800 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:42.797+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:43.164 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:43.160+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:43.184 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:43.179+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:43.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:43.289+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:43.348 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:43.343+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:43.601 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:43.597+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:43.676 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:43.672+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:43.838 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:43.835+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:44.156 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:44.152+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:44.178 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:44.173+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:44.290 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:44.285+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:44.399 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:44.393+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:44.631 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:44.627+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:44.719 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:44.715+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:44.804 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:44.800+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:45.131 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:45.126+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:45.172 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:45.167+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:45.250 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:45.246+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:45.375 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:45.369+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:45.590 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:45.585+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:45.725 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:45.720+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:45.784 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:45.780+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:46.131 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:46.127+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:46.202 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:46.198+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:46.235 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:46.231+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:46.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:46.388+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:46.618 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:46.614+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:46.718 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:46.714+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:46.765 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:46.761+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:47.126 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:47.121+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:47.168 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:47.164+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:47.228 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:47.220+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:47.276 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:02:47.271+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 41sec e428 v428) 2019-12-09T20:02:47.425 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:47.420+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:47.620 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:47.615+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:47.688 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:47.683+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:47.760 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:47.756+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:48.156 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:48.151+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:48.164 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:48.157+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:48.271 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:48.267+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:48.393 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:48.388+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:48.627 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:48.622+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:48.692 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:48.688+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:48.747 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:48.742+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:49.112 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:49.107+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:49.190 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:49.185+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:49.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:49.296+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:49.424 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:49.420+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:49.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:49.614+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:49.742 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:49.739+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:49.761 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:49.758+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:50.083 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:50.078+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:50.192 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:50.187+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:50.303 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:50.299+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:50.435 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:50.431+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:50.569 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:50.566+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:50.750 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:50.746+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:50.769 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:50.764+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:51.060 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:51.057+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:51.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:51.205+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:51.304 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:51.300+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:51.394 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:51.389+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:51.565 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:51.561+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:51.784 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:51.780+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:51.785 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:51.781+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:52.052 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:52.047+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:52.213 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:52.209+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:52.292 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:02:52.289+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 41sec e428 v428) 2019-12-09T20:02:52.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:52.313+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:52.361 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:52.357+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:52.546 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:52.543+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:52.785 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:52.782+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:52.792 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:52.789+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:53.041 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:53.037+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:53.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:53.221+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:53.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:53.311+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:53.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:53.360+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:53.553 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:53.549+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:53.784 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:53.779+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:53.804 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:53.800+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:54.024 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:54.020+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:54.224 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:54.219+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:54.336 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:54.331+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:54.353 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:54.348+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:54.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:54.517+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:54.764 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:54.760+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:54.798 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:54.794+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:55.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:55.028+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:55.243 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:55.238+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:55.339 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:55.332+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:55.357 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:55.353+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:55.539 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:55.535+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:55.777 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:55.773+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:55.813 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:55.809+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:56.021 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:56.016+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:56.266 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:56.262+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:56.315 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:56.311+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:56.349 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:56.345+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:56.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:56.517+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:02:56.760 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:02:56.755+0000 1694f700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.432394+0000 front 2019-12-09T20:01:30.714525+0000 (oldest deadline 2019-12-09T20:02:15.950318+0000) 2019-12-09T20:02:56.825 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:02:56.821+0000 1694f700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.242655+0000 front 2019-12-09T20:01:31.964052+0000 (oldest deadline 2019-12-09T20:02:12.244238+0000) 2019-12-09T20:02:56.987 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:02:56.981+0000 1694f700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:32.166633+0000 front 2019-12-09T20:01:32.097547+0000 (oldest deadline 2019-12-09T20:02:17.583649+0000) 2019-12-09T20:02:57.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:02:57.266+0000 1694f700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.151256+0000 front 2019-12-09T20:01:29.149804+0000 (oldest deadline 2019-12-09T20:02:14.234665+0000) 2019-12-09T20:02:57.317 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:02:57.312+0000 1694f700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:31.992574+0000 front 2019-12-09T20:01:31.994989+0000 (oldest deadline 2019-12-09T20:02:16.442326+0000) 2019-12-09T20:02:57.354 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:02:57.349+0000 1694f700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:29.995846+0000 front 2019-12-09T20:01:30.087419+0000 (oldest deadline 2019-12-09T20:02:13.089946+0000) 2019-12-09T20:02:57.362 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:02:57.357+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.106:6810/9717,v1:172.21.15.106:6811/9717] for 41sec e428 v428) 2019-12-09T20:02:57.549 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:02:57.541+0000 1694f700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.106:6814 osd.4 since back 2019-12-09T20:01:30.066974+0000 front 2019-12-09T20:01:29.997139+0000 (oldest deadline 2019-12-09T20:02:13.459650+0000) 2019-12-09T20:03:27.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:03:27.862+0000 1694f700 -1 osd.2 430 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4385.0:121701 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821402496 gen 126] snapc 0=[] ondisk+write+known_if_redirected e429) 2019-12-09T20:03:37.754 INFO:tasks.ceph.osd.4.smithi106.stderr:*** Caught signal (Segmentation fault) ** 2019-12-09T20:03:37.754 INFO:tasks.ceph.osd.4.smithi106.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:03:37.860 INFO:tasks.ceph.osd.4.smithi106.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:03:37.860 INFO:tasks.ceph.osd.4.smithi106.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:03:37.860 INFO:tasks.ceph.osd.4.smithi106.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:03:37.861 INFO:tasks.ceph.osd.4.smithi106.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:03:37.862 INFO:tasks.ceph.osd.4.smithi106.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:03:37.862 INFO:tasks.ceph.osd.4.smithi106.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:03:37.862 INFO:tasks.ceph.osd.4.smithi106.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:03:37.862 INFO:tasks.ceph.osd.4.smithi106.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:03:38.264 INFO:tasks.ceph.osd.4.smithi106.stderr:2019-12-09T20:03:38.015+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:03:38.264 INFO:tasks.ceph.osd.4.smithi106.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:03:38.264 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:38.264 INFO:tasks.ceph.osd.4.smithi106.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:03:38.264 INFO:tasks.ceph.osd.4.smithi106.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:03:38.265 INFO:tasks.ceph.osd.4.smithi106.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:03:38.266 INFO:tasks.ceph.osd.4.smithi106.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:03:38.267 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:39.246 INFO:tasks.ceph.osd.4.smithi106.stderr: -2183> 2019-12-09T20:03:38.015+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:03:39.246 INFO:tasks.ceph.osd.4.smithi106.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:03:39.246 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:39.246 INFO:tasks.ceph.osd.4.smithi106.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:03:39.247 INFO:tasks.ceph.osd.4.smithi106.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:03:39.248 INFO:tasks.ceph.osd.4.smithi106.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:03:39.249 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:39.613 INFO:tasks.ceph.osd.4.smithi106.stderr: -2227> 2019-12-09T20:03:38.015+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:03:39.613 INFO:tasks.ceph.osd.4.smithi106.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:03:39.613 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:39.613 INFO:tasks.ceph.osd.4.smithi106.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:03:39.614 INFO:tasks.ceph.osd.4.smithi106.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:03:39.615 INFO:tasks.ceph.osd.4.smithi106.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:03:39.616 INFO:tasks.ceph.osd.4.smithi106.stderr: 2019-12-09T20:03:42.950 INFO:tasks.ceph.osd.4.smithi106.stderr:daemon-helper: command crashed with signal 11 2019-12-09T20:03:49.277 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2019-12-09T20:03:55.293 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:03:55.288+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:03:55.589 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~6s 2019-12-09T20:03:55.737 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:03:55.732+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:03:56.265 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:03:56.260+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:03:56.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:03:56.358+0000 1694f700 -1 osd.2 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:10.715896+0000 front 2019-12-09T20:03:10.672324+0000 (oldest deadline 2019-12-09T20:03:55.337960+0000) 2019-12-09T20:03:56.732 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:03:56.726+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:03:57.140 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:03:57.135+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:03:57.320 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:03:57.313+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:03:57.327 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:03:57.324+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:03:57.377 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:03:57.371+0000 1694f700 -1 osd.2 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:10.715896+0000 front 2019-12-09T20:03:10.672324+0000 (oldest deadline 2019-12-09T20:03:55.337960+0000) 2019-12-09T20:03:57.769 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:03:57.765+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:03:58.102 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:03:58.094+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:03:58.341 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:03:58.337+0000 1694f700 -1 osd.2 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:10.715896+0000 front 2019-12-09T20:03:10.672324+0000 (oldest deadline 2019-12-09T20:03:55.337960+0000) 2019-12-09T20:03:58.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:03:58.376+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:03:58.382 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:03:58.377+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:03:58.785 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:03:58.780+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:03:58.874 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:03:58.870+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:03:59.104 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:03:59.100+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:03:59.341 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:03:59.336+0000 1694f700 -1 osd.2 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:10.715896+0000 front 2019-12-09T20:03:10.672324+0000 (oldest deadline 2019-12-09T20:03:55.337960+0000) 2019-12-09T20:03:59.344 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:03:59.336+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:03:59.352 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:03:59.347+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:03:59.838 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:03:59.832+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:03:59.920 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:03:59.915+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:00.065 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:00.060+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:00.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:00.342+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:00.359 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:00.354+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:00.817 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:00.813+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:00.914 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:00.911+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:01.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:01.039+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:01.307 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:01.301+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:01.408 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:01.404+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:01.822 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:01.817+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:01.901 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~13s 2019-12-09T20:04:01.950 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:01.945+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:02.067 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:02.062+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:02.358 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:02.354+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:02.412 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:02.407+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:02.874 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:02.870+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:02.906 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:02.902+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:03.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:03.107+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:03.363 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:03.359+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:03.409 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:03.398+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:03.890 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:03.886+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:03.895 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:03.891+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:04.089 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:04.084+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:04.411 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:04.406+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:04.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:04.447+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:04.862 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:04.857+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:04.867 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:04.862+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:05.129 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:05.124+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:05.399 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:05.395+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:05.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:05.424+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:05.839 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:05.835+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:05.847 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:05.843+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:06.094 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:06.088+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:06.402 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:06.398+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:06.454 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:06.449+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:06.821 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:06.815+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:06.829 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:06.826+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:07.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:07.106+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:07.404 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:07.399+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:07.446 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:07.440+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:07.798 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:07.794+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:07.799 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:07.794+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:08.159 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:08.155+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:08.210 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~19s 2019-12-09T20:04:08.417 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:08.413+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:08.432 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:08.427+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:08.763 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:08.759+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:08.823 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:08.820+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:09.181 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:09.175+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:09.460 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:09.455+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:09.467 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:09.462+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:09.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:09.797+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:09.803 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:09.799+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:10.207 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:10.202+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:10.448 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:10.443+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:10.511 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:10.507+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:10.763 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:10.759+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:10.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:10.795+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:11.226 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:11.220+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:11.469 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:11.464+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:11.493 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:11.489+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:11.723 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:11.718+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:11.823 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:11.818+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:12.259 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:12.253+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:12.432 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:12.427+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:12.478 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:12.474+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:12.733 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:12.730+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:12.792 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:12.787+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:13.235 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:13.228+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:13.425 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:13.420+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:13.478 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:13.474+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:13.764 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:13.760+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:13.833 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:13.828+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:14.231 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:14.227+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:14.432 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:14.428+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:14.450 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:14.445+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:14.520 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~25s 2019-12-09T20:04:14.791 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:14.788+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:14.845 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:14.840+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:15.219 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:15.214+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:15.434 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:15.429+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:15.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:15.439+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:15.767 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:15.762+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:15.841 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:15.835+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:16.175 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:16.171+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:16.439 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:16.435+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:16.458 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:16.453+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:16.744 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:16.740+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:16.811 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:16.806+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:17.229 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:17.224+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:17.393 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:17.389+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:17.452 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:17.447+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:17.790 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:17.786+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:17.830 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:17.825+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:18.191 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:18.186+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:18.417 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:18.412+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:18.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:18.435+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:18.774 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:18.770+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:18.823 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:18.820+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:19.231 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:19.227+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:19.380 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:19.375+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:19.469 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:19.464+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:19.727 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:19.723+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:19.820 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:19.816+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:20.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:20.192+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:20.340 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:20.334+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:20.472 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:20.468+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:20.695 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:20.691+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:20.791 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:20.786+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:20.830 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~32s 2019-12-09T20:04:21.158 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:21.151+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:21.307 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:21.302+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:21.490 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:21.486+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:21.708 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:21.703+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:21.777 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:21.772+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:22.168 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:04:22.162+0000 1694f700 -1 osd.5 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:12.104507+0000 front 2019-12-09T20:03:11.838872+0000 (oldest deadline 2019-12-09T20:03:56.459890+0000) 2019-12-09T20:04:22.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:04:22.285+0000 1694f700 -1 osd.6 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:11.160159+0000 front 2019-12-09T20:03:11.243521+0000 (oldest deadline 2019-12-09T20:03:55.165422+0000) 2019-12-09T20:04:22.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:04:22.524+0000 1694f700 -1 osd.1 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:13.588179+0000 front 2019-12-09T20:03:13.796560+0000 (oldest deadline 2019-12-09T20:03:57.077338+0000) 2019-12-09T20:04:22.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:04:22.665+0000 1694f700 -1 osd.0 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.011314+0000 front 2019-12-09T20:03:14.026745+0000 (oldest deadline 2019-12-09T20:03:58.676040+0000) 2019-12-09T20:04:22.745 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:04:22.740+0000 1694f700 -1 osd.7 431 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:03:14.290846+0000 front 2019-12-09T20:03:14.468679+0000 (oldest deadline 2019-12-09T20:03:55.378583+0000) 2019-12-09T20:04:27.141 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~38s 2019-12-09T20:04:33.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~44s 2019-12-09T20:04:39.768 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~50s 2019-12-09T20:04:46.080 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~57s 2019-12-09T20:04:52.392 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~63s 2019-12-09T20:04:58.705 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~69s 2019-12-09T20:05:05.015 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~76s 2019-12-09T20:05:11.328 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~82s 2019-12-09T20:05:12.704 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:12.668+0000 30f85700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:12.719 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:12.711+0000 31f87700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:12.736 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:12.723+0000 35f8f700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:13.026 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:12.934+0000 31786700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:13.438 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:13.203+0000 36790700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:13.560 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:13.347+0000 33f8b700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:13.950 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T20:05:13.935+0000 20e25700 -1 RGWWatcher::handle_error cookie 1544508640 err (107) Transport endpoint is not connected 2019-12-09T20:05:14.016 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:14.001+0000 32f89700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:16.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:05:16.046+0000 36f91700 -1 osd.3 437 build_incremental_map_msg missing incremental map 437 2019-12-09T20:05:17.643 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~88s 2019-12-09T20:05:23.956 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~95s 2019-12-09T20:05:30.269 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~101s 2019-12-09T20:05:36.582 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~107s 2019-12-09T20:05:42.894 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~114s 2019-12-09T20:05:49.207 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~120s 2019-12-09T20:05:55.518 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~126s 2019-12-09T20:06:01.832 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~133s 2019-12-09T20:06:08.268 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~139s 2019-12-09T20:06:14.578 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~145s 2019-12-09T20:06:20.888 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~152s 2019-12-09T20:06:27.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~158s 2019-12-09T20:06:33.512 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~164s 2019-12-09T20:06:39.827 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~171s 2019-12-09T20:06:46.137 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~177s 2019-12-09T20:06:52.447 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~183s 2019-12-09T20:06:57.532 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:06:57.527+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:06:57.683 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:06:57.678+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:06:58.584 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:06:58.580+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:06:58.701 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:06:58.696+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:06:58.760 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~189s 2019-12-09T20:06:59.477 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:06:59.473+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:06:59.499 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:06:59.495+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:06:59.538 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:06:59.533+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:06:59.752 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:06:59.746+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:00.479 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:00.475+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:00.498 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:00.493+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:00.531 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:00.524+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:00.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:00.798+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:00.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:00.944+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:01.469 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:01.465+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:01.538 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:01.532+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:01.550 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:01.544+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:01.834 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:01.829+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:01.894 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:01.889+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:01.980 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:01.977+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:02.032 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:45:53.881 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.044 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:45:53.894 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.054 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:45:53.872 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.054 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:45:53.875 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.054 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:45:53.879 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.063 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:45:53.887 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.272 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:45:54.097 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.312 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:45:54.162 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.392 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:45:54.216 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:07:02.443 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:02.439+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:02.521 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:02.516+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:02.565 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:02.561+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:02.879 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:02.874+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:02.939 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:02.935+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:02.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:02.992+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:03.410 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:03.405+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:03.483 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:03.480+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:03.567 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:03.564+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:03.928 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:03.923+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:03.955 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:03.951+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:04.021 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:04.018+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:04.438 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:04.432+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:04.443 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:04.436+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:04.580 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:04.573+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:04.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:04.926+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:04.939 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:04.934+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:05.057 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:05.052+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:05.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~196s 2019-12-09T20:07:05.432 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:05.429+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:05.485 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:05.479+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:05.545 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:05.541+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:05.943 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:05.937+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:05.959 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:05.953+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:06.055 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:06.050+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:06.430 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:06.426+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:06.505 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:06.499+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:06.535 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:06.531+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:06.920 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:06.916+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:06.959 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:06.954+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:07.015 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:07.010+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:07.435 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:07.429+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:07.467 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:07.461+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:07.579 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:07.574+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:07.938 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:07.933+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:07.949 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:07.944+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:07.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:07.992+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:08.442 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:08.437+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:08.489 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:08.484+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:08.545 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:08.541+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:08.910 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:08.906+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:08.944 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:08.939+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:09.008 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:09.003+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:09.456 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:09.451+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:09.497 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:09.490+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:09.595 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:09.591+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:09.901 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:09.896+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:09.968 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:09.963+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:09.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:09.969+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:10.461 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:10.456+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:10.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:10.524+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:10.575 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:10.571+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:10.908 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:10.903+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:10.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:10.956+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:10.977 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:10.972+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:11.386 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~202s 2019-12-09T20:07:11.493 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:11.487+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:11.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:11.509+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:11.547 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:11.541+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:11.884 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:11.879+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:12.002 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:11.997+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:12.013 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:12.007+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:12.475 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:12.470+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:12.531 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:12.522+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:12.566 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:12.561+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:12.909 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:12.905+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:13.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:13.012+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:13.051 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:13.045+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:13.492 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:13.487+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:13.563 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:13.556+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:13.606 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:13.601+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:13.914 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:13.909+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:13.983 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:13.977+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:14.049 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:14.044+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:14.545 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:14.528+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:14.586 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:14.581+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:14.647 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:14.643+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:14.909 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:14.899+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:14.977 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:14.971+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:15.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:15.091+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:15.530 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:15.525+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:15.594 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:15.589+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:15.619 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:15.613+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:15.872 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:15.867+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:16.011 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:16.005+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:16.096 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:16.090+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:16.552 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:16.548+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:16.559 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:16.554+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:16.648 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:16.643+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:16.894 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:16.888+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:17.058 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:17.053+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:17.136 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:17.130+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:17.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:17.528+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:17.534 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:17.528+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:17.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:17.606+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:17.695 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~208s 2019-12-09T20:07:17.847 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:17.843+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:18.023 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:18.018+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:18.174 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:18.169+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:18.504 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:18.500+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:18.559 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:18.554+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:18.624 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:18.619+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:18.859 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:18.851+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:19.018 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:19.013+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:19.218 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:19.213+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:19.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:19.544+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:19.578 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:19.574+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:19.589 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:19.583+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:19.866 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:19.861+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:19.994 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:19.988+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:20.261 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:20.255+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:20.537 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:20.532+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:20.593 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:20.588+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:20.595 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:20.591+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:20.848 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:20.842+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:21.048 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:21.041+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:21.240 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:07:21.234+0000 1694f700 -1 osd.0 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.382553+0000 front 2019-12-09T20:06:16.381298+0000 (oldest deadline 2019-12-09T20:07:00.206299+0000) 2019-12-09T20:07:21.552 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:21.547+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:21.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:21.564+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:21.595 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:07:21.590+0000 1694f700 -1 osd.7 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.883378+0000 front 2019-12-09T20:06:15.947796+0000 (oldest deadline 2019-12-09T20:06:56.961244+0000) 2019-12-09T20:07:21.835 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:21.831+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:22.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:07:22.028+0000 1694f700 -1 osd.6 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.471318+0000 front 2019-12-09T20:06:15.300068+0000 (oldest deadline 2019-12-09T20:07:01.175677+0000) 2019-12-09T20:07:22.556 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:22.551+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:22.599 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:22.595+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:22.619 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:07:22.615+0000 1694f700 -1 osd.1 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:16.559622+0000 front 2019-12-09T20:06:16.495996+0000 (oldest deadline 2019-12-09T20:06:58.602966+0000) 2019-12-09T20:07:22.879 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:22.873+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:23.545 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:23.540+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:23.594 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:23.589+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:23.877 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:07:23.872+0000 1694f700 -1 osd.5 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:14.434134+0000 front 2019-12-09T20:06:14.597942+0000 (oldest deadline 2019-12-09T20:06:57.469171+0000) 2019-12-09T20:07:24.020 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~215s 2019-12-09T20:07:24.566 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:24.559+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:24.589 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:24.579+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:25.576 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:25.569+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:25.610 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:07:25.596+0000 1694f700 -1 osd.2 437 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:06:15.754520+0000 front 2019-12-09T20:06:15.475714+0000 (oldest deadline 2019-12-09T20:06:59.126989+0000) 2019-12-09T20:07:26.596 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:26.592+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:27.558 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:27.553+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:28.514 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:28.509+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:29.556 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:29.551+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:30.333 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~221s 2019-12-09T20:07:30.523 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:30.518+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:31.482 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:31.478+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:32.506 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:32.502+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:33.557 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:33.553+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:34.596 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:34.592+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:35.642 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:35.639+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:36.645 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~227s 2019-12-09T20:07:36.672 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:36.668+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:37.710 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:37.707+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:38.665 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:38.662+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:39.710 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:39.707+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:40.749 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:40.744+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:41.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:41.707+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:42.734 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:42.731+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:42.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~234s 2019-12-09T20:07:43.724 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:43.717+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:44.751 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:44.747+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:45.761 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:45.758+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:46.770 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:46.766+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:47.787 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:47.781+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:48.789 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:48.785+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:49.284 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~240s 2019-12-09T20:07:49.758 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:49.754+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:50.809 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:50.806+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:51.787 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:51.783+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:52.807 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:52.802+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:53.796 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:53.790+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:54.781 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:54.776+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:55.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~246s 2019-12-09T20:07:55.755 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:55.751+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:56.779 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:56.775+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:57.752 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:57.746+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:58.708 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:58.704+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:07:59.711 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:07:59.708+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:00.668 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:00.664+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:01.716 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:01.713+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:01.905 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~253s 2019-12-09T20:08:02.745 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:02.742+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:03.702 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:03.699+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:04.710 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:04.707+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:05.718 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:05.714+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:06.708 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:06.703+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:07.750 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:07.746+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:08.219 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~259s 2019-12-09T20:08:08.775 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:08.772+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:09.831 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:09.828+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:10.861 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:10.858+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:11.848 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:11.844+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:12.885 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:12.882+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:13.923 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:13.920+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:14.539 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~265s 2019-12-09T20:08:14.941 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:14.937+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:15.934 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:15.931+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:16.968 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:16.965+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:17.943 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:17.939+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:18.954 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:18.951+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:19.908 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:19.905+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:20.849 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~272s 2019-12-09T20:08:20.954 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:20.950+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:21.927 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:21.924+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:22.906 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:22.902+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:23.895 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:23.891+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:24.942 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:24.937+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:25.932 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:25.928+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:26.972 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:26.967+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:27.163 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~278s 2019-12-09T20:08:28.020 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:28.016+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:29.007 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:29.003+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:29.989 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:29.978+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:31.012 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:31.008+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:32.048 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:32.043+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:33.029 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:33.025+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:33.477 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~284s 2019-12-09T20:08:34.075 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:34.070+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:35.108 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:35.102+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:36.107 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:36.102+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:37.146 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:37.140+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:38.130 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:38.126+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:39.089 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:39.085+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:39.786 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~291s 2019-12-09T20:08:40.077 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:40.072+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:41.085 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:41.080+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:42.128 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:42.123+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:43.173 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:43.169+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:44.152 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:44.147+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:45.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:45.186+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:46.101 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~297s 2019-12-09T20:08:46.203 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:46.199+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:47.176 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:47.170+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:48.139 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:48.136+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:49.161 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:49.158+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:50.159 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:50.155+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:51.131 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:51.128+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:52.180 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:52.176+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:52.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~303s 2019-12-09T20:08:52.425 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2019-12-09T20:08:52.425 ERROR:tasks.daemonwatchdog.daemon_watchdog:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 38, in _run self.watch() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 114, in watch self.bark() File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-ofriedma-dynamic-disable/qa/tasks/daemonwatchdog.py", line 54, in bark for mount in self.ctx.mounts.values(): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/config.py", line 253, in __getattr__ raise AttributeError(name) AttributeError: mounts 2019-12-09T20:08:53.217 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:53.210+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:54.222 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:54.219+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:55.200 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:55.196+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:56.232 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:56.228+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:57.269 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:57.266+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:58.298 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:58.295+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:08:59.333 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:08:59.330+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:00.372 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:00.368+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:01.368 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:01.364+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:02.388 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:02.384+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:03.388 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:03.384+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:04.356 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:04.351+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:05.356 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:05.354+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:06.366 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:06.363+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:07.383 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:07.380+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:08.338 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:08.335+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:09.351 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:09.348+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:10.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:10.311+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:11.271 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:11.268+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:12.271 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:12.267+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:13.269 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:13.266+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:14.308 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:14.304+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:15.342 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:15.338+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:16.394 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:16.390+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:17.424 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:17.420+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:18.409 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:18.405+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:19.443 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:19.440+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:20.463 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:20.459+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:21.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:21.494+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:22.501 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:22.491+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:23.450 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:23.447+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:24.403 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:48:16.252 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.416 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:48:16.266 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.469 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:48:16.294 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.476 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:48:16.301 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:48:16.312 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.496 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:48:16.321 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.497 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:24.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:24.621 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:48:16.446 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.643 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:48:16.493 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:24.685 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:48:16.509 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:09:25.536 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:25.531+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:26.564 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:26.560+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:27.538 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:27.534+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:28.495 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:28.491+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:29.484 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:29.481+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:30.481 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:30.477+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:31.472 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:31.467+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:32.466 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:32.460+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:33.424 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:33.420+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:34.435 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:34.431+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:35.476 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:35.472+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:36.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:36.505+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:37.512 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:37.509+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:38.525 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:38.520+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:39.509 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:39.505+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:40.530 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:40.525+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:41.539 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:41.535+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:42.513 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:42.509+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:43.491 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:43.487+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:44.538 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:44.534+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:45.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:45.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:46.515 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:46.510+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:47.551 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:47.546+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:48.546 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:48.541+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:49.534 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:49.530+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:50.522 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:50.519+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:51.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:51.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:52.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:52.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:53.548 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:53.543+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:54.600 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:54.596+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:55.580 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:55.575+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:56.561 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:56.556+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:57.541 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:57.536+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:58.516 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:58.511+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:09:59.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:09:59.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:00.505 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:00.501+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:01.559 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:01.554+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:02.528 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:02.523+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:03.536 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:03.533+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:04.515 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:04.511+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:05.498 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:05.493+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:06.505 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:06.501+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:07.510 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:07.506+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:08.562 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:08.559+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:09.587 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:09.584+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:10.599 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:10.596+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:11.644 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:11.640+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:12.671 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:12.667+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:13.718 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:13.714+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:14.755 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:14.752+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:15.805 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:15.802+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:16.807 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:16.804+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:17.779 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:17.776+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:18.789 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:18.783+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:19.791 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:19.788+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:20.833 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:20.829+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:21.869 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:21.863+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:22.913 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:22.903+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:23.862 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:23.858+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:24.907 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:24.905+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:25.889 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:25.885+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:26.873 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:26.870+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:27.893 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:27.888+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:28.885 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:28.879+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:29.917 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:29.914+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:30.953 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:30.950+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:31.954 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:31.951+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:32.995 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:32.992+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:34.049 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:34.046+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:35.014 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:35.002+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:35.997 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:35.993+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:36.995 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:36.992+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:37.972 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:37.969+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:39.003 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:39.000+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:40.057 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:40.053+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:41.013 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:41.005+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:42.008 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:42.005+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:43.049 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:43.045+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:44.056 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:44.053+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:45.071 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:45.066+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:46.053 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:46.048+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:47.069 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:47.066+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:48.082 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:48.078+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:49.115 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:49.112+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:50.105 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:50.101+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:51.131 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:51.127+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:52.168 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:52.164+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:53.155 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:53.152+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:54.120 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:54.115+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:55.115 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:55.108+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:56.101 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:56.091+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:57.137 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:57.131+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:58.186 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:58.181+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:10:59.176 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:10:59.171+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:00.225 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:00.221+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:01.261 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:01.256+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:02.306 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:02.302+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:03.343 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:03.338+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:04.318 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:04.314+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:05.310 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:05.305+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:06.285 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:06.281+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:07.314 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:07.310+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:08.333 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:08.329+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:09.377 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:09.372+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:10.331 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:10.327+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:11.089 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:50:02.940 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.100 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:50:02.950 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.112 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:50:02.937 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.115 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:50:02.940 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.116 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:50:02.941 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.122 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:50:02.947 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.239 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:50:03.063 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.294 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:50:03.119 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.303 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:50:03.152 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:11:11.325 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:11.321+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:12.334 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:12.331+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:13.325 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:13.321+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:14.342 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:14.338+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:15.395 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:15.391+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:16.379 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:16.373+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:17.428 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:17.424+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:18.444 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:18.441+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:19.403 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:19.398+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:20.437 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:20.434+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:21.401 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:21.397+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:22.379 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:22.375+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:23.345 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:23.341+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:24.395 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:24.392+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:25.370 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:25.367+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:26.382 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:26.377+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:27.367 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:27.363+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:28.397 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:28.383+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:29.345 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:29.342+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:30.315 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:30.303+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:31.296 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:31.293+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:32.330 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:32.327+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:33.378 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:33.374+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:34.338 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:34.335+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:35.353 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:35.349+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:36.336 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:36.333+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:37.346 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:37.343+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:38.324 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:38.321+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:39.298 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:39.294+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:40.281 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:40.276+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:41.275 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:41.272+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:42.310 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:42.307+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:43.347 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:43.343+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:44.352 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:44.348+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:45.400 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:45.397+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:46.446 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:46.443+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:47.494 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:47.490+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:48.530 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:48.526+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:49.500 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:49.495+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:50.526 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:50.523+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:51.554 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:51.551+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:52.580 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:52.577+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:53.539 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:53.534+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:54.576 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:54.573+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:55.597 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:55.593+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:56.645 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:56.639+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:57.639 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:57.636+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:58.595 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:58.592+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:11:59.612 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:11:59.608+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:00.616 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:00.612+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:01.586 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:01.583+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:02.552 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:02.548+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:03.572 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:03.569+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:04.559 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:04.556+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:05.560 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:05.556+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:06.582 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:06.578+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:07.595 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:07.592+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:08.596 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:08.591+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:09.580 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:09.575+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:10.628 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:10.623+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:11.623 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:11.618+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:12.646 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:12.640+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:13.646 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:13.641+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:14.668 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:14.662+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:15.675 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:15.670+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:16.646 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:16.641+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:17.696 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:17.692+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:18.683 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:18.678+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:19.641 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:19.637+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:20.634 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:20.628+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:21.613 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:21.608+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:22.647 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:22.642+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:23.698 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:23.689+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:24.665 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:24.661+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:25.674 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:25.669+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:26.650 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:26.645+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:27.617 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:27.613+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:28.655 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:28.651+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:29.709 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:29.705+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:30.683 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:30.678+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:31.703 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:31.698+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:32.697 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:32.692+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:33.671 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:33.668+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:34.724 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:34.719+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:35.721 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:35.718+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:36.687 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:36.684+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:37.723 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:37.718+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:38.734 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:38.730+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:39.774 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:39.770+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:40.781 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:40.778+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:41.821 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:41.818+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:42.861 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:42.857+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:43.841 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:43.837+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:44.874 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:44.871+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:45.920 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:45.916+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:46.935 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:46.932+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:47.926 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:47.923+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:48.941 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:48.938+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:49.952 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:49.945+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:50.914 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:50.909+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:51.949 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:51.946+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:52.985 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:52.981+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:53.949 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:53.946+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:54.992 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:54.986+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:56.022 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:56.018+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:57.016 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:57.012+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:58.030 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:58.024+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:12:59.008 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:12:59.005+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:00.014 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:00.010+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:01.048 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:01.044+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:02.053 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:02.049+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:03.079 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:03.073+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:04.054 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:04.048+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:05.043 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:05.041+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:06.066 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:06.063+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:07.076 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:07.072+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:08.078 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:08.073+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:09.066 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:09.063+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:10.045 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:10.041+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:11.090 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:11.086+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:12.140 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:12.134+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:13.149 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:13.145+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:14.170 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:14.166+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:15.206 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:15.202+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:16.173 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:16.169+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:17.144 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:17.140+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:18.173 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:18.170+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:19.156 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:19.153+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:20.130 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:20.127+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:21.172 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:21.169+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:22.151 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:22.147+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:23.177 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:23.174+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:24.191 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:24.188+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:25.197 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:25.192+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:26.174 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:26.169+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:27.193 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:27.188+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:28.243 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:28.238+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:29.224 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:29.219+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:30.278 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:30.272+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:31.256 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:31.251+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:32.266 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:32.260+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:33.237 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:33.233+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:34.219 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:34.214+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:35.217 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:35.212+0000 1694f700 -1 osd.3 437 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:36.348 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:36.305+0000 1694f700 -1 osd.3 438 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:37.457 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:37.443+0000 1694f700 -1 osd.3 439 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:38.779 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:38.648+0000 1694f700 -1 osd.3 440 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:39.797 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:39.787+0000 1694f700 -1 osd.3 440 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:40.928 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:40.901+0000 1694f700 -1 osd.3 441 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4458.0:119482 7.4 7:2c0c0c23:reshard::reshard.0000000001:head [call lock.lock in=70b] snapc 0=[] RETRY=79 ondisk+retry+write+known_if_redirected e435) 2019-12-09T20:13:43.956 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:43.952+0000 32788700 -1 osd.3 442 build_incremental_map_msg missing incremental map 404 2019-12-09T20:13:43.965 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:43.961+0000 32788700 -1 osd.3 442 build_incremental_map_msg missing incremental map 416 2019-12-09T20:13:43.967 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:43.963+0000 32788700 -1 osd.3 442 build_incremental_map_msg missing incremental map 420 2019-12-09T20:13:43.982 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:13:43.976+0000 32788700 -1 osd.3 442 build_incremental_map_msg missing incremental map 437 2019-12-09T20:14:19.075 INFO:teuthology.orchestra.run.smithi174.stderr:testUnorderedList (test.functional.tests.TestContainer) ... ok 2019-12-09T20:14:39.740 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:53:31.591 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:39.751 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:53:31.602 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:39.766 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:53:31.590 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:39.769 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:53:31.593 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:39.769 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:53:31.594 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:39.776 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:53:31.600 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:40.069 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:53:31.894 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:40.085 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:53:31.936 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:14:40.133 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:53:31.958 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:15:54.401 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:54.396+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:55.053 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:15:55.048+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:15:55.307 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:15:55.301+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:15:55.356 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:55.351+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:55.525 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:15:55.516+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:15:55.552 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:15:55.547+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:15:56.061 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:15:56.056+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:15:56.333 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:15:56.327+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:15:56.381 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:56.377+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:56.513 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:15:56.507+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:15:56.544 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:15:56.537+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:15:57.058 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:15:57.052+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:15:57.349 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:15:57.344+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:15:57.410 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:57.403+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:57.549 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:15:57.543+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:15:57.565 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:15:57.558+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:15:57.993 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:15:57.985+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:15:58.032 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:15:58.027+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:15:58.380 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:15:58.375+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:15:58.382 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:58.377+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:58.526 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:15:58.521+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:15:58.550 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:15:58.544+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:15:58.994 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:15:58.987+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:15:59.047 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:15:59.040+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:15:59.381 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:15:59.376+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:15:59.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:15:59.383+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:15:59.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:15:59.528+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:15:59.565 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:15:59.558+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:15:59.974 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:15:59.969+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:00.022 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:00.017+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:00.350 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:00.345+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:00.393 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:00.388+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:00.522 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:00.517+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:00.536 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:00.529+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:00.988 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:00.982+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:01.028 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:01.023+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:01.348 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:01.344+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:01.371 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:01.367+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:01.485 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:01.479+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:01.517 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:54:53.367 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.519 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:54:53.369 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.520 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:01.515+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:01.573 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:54:53.398 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.574 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:54:53.399 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.577 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:54:53.402 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.585 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:54:53.410 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.614 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:54:53.464 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.661 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:54:53.486 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:01.688 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:54:53.513 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:16:02.029 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:02.026+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:02.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:02.028+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:02.350 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:02.347+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:02.414 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:02.411+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:02.510 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:02.506+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:02.546 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:02.541+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:03.013 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:03.010+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:03.082 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:03.077+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:03.365 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:03.361+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:03.442 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:03.438+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:03.492 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:03.486+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:03.512 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:03.506+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:04.042 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:04.038+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:04.051 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:04.046+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:04.361 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:04.358+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:04.436 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:04.431+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:04.484 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:04.479+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:04.489 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:04.483+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:05.001 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:04.997+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:05.074 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:05.070+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:05.358 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:05.346+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:05.458 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:05.452+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:05.465 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:05.459+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:05.500 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:05.495+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:06.027 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:06.022+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:06.127 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:06.122+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:06.389 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:06.379+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:06.460 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:06.456+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:06.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:06.477+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:06.549 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:06.541+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:06.984 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:06.979+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:07.139 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:07.133+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:07.378 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:07.373+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:07.440 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:07.433+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:07.506 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:07.501+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:07.515 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:07.510+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:08.007 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:08.000+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:08.134 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:08.129+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:08.399 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:08.394+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:08.477 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:08.473+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:08.478 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:08.473+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:08.499 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:08.494+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:09.027 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:09.023+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:09.106 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:09.100+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:09.417 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:09.412+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:09.444 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:09.440+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:09.517 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:09.512+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:09.529 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:09.524+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:09.990 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:09.985+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:10.064 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:10.060+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:10.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:10.423+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:10.463 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:10.458+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:10.532 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:10.525+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:10.558 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:10.554+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:11.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:11.030+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:11.055 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:11.050+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:11.413 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:11.408+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:11.420 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:11.414+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:11.517 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:11.512+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:11.567 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:11.563+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:11.989 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:11.984+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:12.024 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:12.018+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:12.411 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:12.406+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:12.429 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:12.425+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:12.478 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:12.472+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:12.525 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:12.521+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:12.990 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:12.985+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:13.033 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:13.028+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:13.366 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:13.361+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:13.437 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:13.429+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:13.527 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:13.522+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:13.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:13.562+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:14.010 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:14.006+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:14.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:14.039+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:14.396 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:14.389+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:14.467 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:14.462+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:14.569 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:14.564+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:14.572 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:14.567+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:15.036 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:15.030+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:15.056 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:15.051+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:15.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:15.352+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:15.449 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:15.444+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:15.587 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:15.583+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:15.597 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:15.592+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:16.081 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:16.076+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:16.088 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:16.083+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:16.350 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:16.345+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:16.496 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:16.491+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:16.569 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:16.564+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:16.620 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:16.616+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:17.065 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:17.060+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:17.099 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:17.094+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:17.336 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:17.331+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:17.497 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:17.492+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:17.567 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:17.562+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:17.632 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:17.626+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:18.082 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:18.076+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:18.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:18.105+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:18.337 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:18.332+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:18.488 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:18.483+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:18.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:18.524+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:18.682 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:18.677+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:19.074 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:19.069+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:19.134 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:19.129+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:19.340 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:19.335+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:19.522 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:19.517+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:19.529 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:19.525+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:19.734 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:19.728+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:20.068 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:20.063+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:20.155 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:20.150+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:20.348 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:20.343+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:20.506 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:20.500+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:20.547 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:20.543+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:20.701 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:20.697+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:21.083 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:21.078+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:21.160 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:21.155+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:21.380 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:21.375+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:21.464 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:21.460+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:21.528 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:21.524+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:21.725 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:21.720+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:22.079 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:22.068+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:22.179 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:22.175+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:22.415 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:22.410+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:22.424 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:22.419+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:22.518 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:22.513+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:22.677 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:22.674+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:23.047 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:23.042+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:23.179 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:23.175+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:23.421 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:23.416+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:23.422 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:23.407+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:23.483 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:23.479+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:23.730 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:23.726+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:24.058 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:24.053+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:24.212 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:24.207+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:24.368 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:24.362+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:24.440 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:24.435+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:24.458 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:24.454+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:24.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:24.719+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:25.082 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:25.077+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:25.218 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:25.213+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:25.384 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:25.378+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:25.427 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:25.422+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:25.450 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:25.446+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:25.762 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:16:25.758+0000 1694f700 -1 osd.2 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:11.095757+0000 front 2019-12-09T20:15:11.154638+0000 (oldest deadline 2019-12-09T20:15:54.574045+0000) 2019-12-09T20:16:26.096 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:16:26.091+0000 1694f700 -1 osd.5 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:14.014112+0000 front 2019-12-09T20:15:13.869305+0000 (oldest deadline 2019-12-09T20:15:54.287643+0000) 2019-12-09T20:16:26.210 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:26.206+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:26.404 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:16:26.399+0000 1694f700 -1 osd.6 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.297730+0000 front 2019-12-09T20:15:13.033307+0000 (oldest deadline 2019-12-09T20:15:53.977422+0000) 2019-12-09T20:16:26.437 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:16:26.431+0000 1694f700 -1 osd.7 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:08.850824+0000 front 2019-12-09T20:15:09.427126+0000 (oldest deadline 2019-12-09T20:15:54.719966+0000) 2019-12-09T20:16:26.452 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:16:26.449+0000 1694f700 -1 osd.1 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:09.412891+0000 front 2019-12-09T20:15:09.646785+0000 (oldest deadline 2019-12-09T20:15:55.219285+0000) 2019-12-09T20:16:27.211 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:16:27.196+0000 1694f700 -1 osd.0 468 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:15:13.312952+0000 front 2019-12-09T20:15:13.215421+0000 (oldest deadline 2019-12-09T20:15:57.166931+0000) 2019-12-09T20:16:33.438 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T20:16:33.427+0000 20e25700 -1 RGWWatcher::handle_error cookie 821402496 err (110) Connection timed out 2019-12-09T20:17:47.168 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:56:39.018 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.172 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:56:39.023 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.198 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:56:39.023 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.207 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:56:39.032 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.215 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:56:39.040 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.229 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:56:39.051 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.388 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:56:39.212 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.410 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:56:39.235 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:47.420 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:56:39.271 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:17:51.073 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:51.068+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:51.643 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:51.638+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:51.959 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:51.955+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:52.041 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:52.036+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:52.093 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:52.088+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:52.687 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:52.681+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:52.958 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:52.954+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:53.037 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:53.032+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:53.086 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:53.075+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:53.679 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:53.675+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:53.747 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:53.742+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:53.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:53.956+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:54.021 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:54.015+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:54.042 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:54.037+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:54.708 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:54.699+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:54.754 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:54.751+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:54.802 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:54.798+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:54.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:54.956+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:55.044 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:55.040+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:55.090 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:55.085+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:55.707 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:55.703+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:55.760 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:55.756+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:55.824 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:55.821+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:55.967 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:55.963+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:56.038 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:56.034+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:56.054 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:56.042+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:56.728 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:56.723+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:56.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:56.785+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:56.805 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:56.802+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:56.931 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:56.926+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:57.073 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:57.068+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:57.082 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:57.078+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:57.695 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:57.690+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:57.819 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:57.812+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:57.838 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:57.835+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:57.947 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:57.942+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:58.104 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:58.098+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:58.120 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:58.116+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:58.734 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:58.729+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:58.810 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:58.805+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:58.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:58.826+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:58.948 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:58.945+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:17:59.113 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:17:59.108+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:17:59.174 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:17:59.169+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:17:59.779 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:17:59.774+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:17:59.801 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:17:59.796+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:17:59.821 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:17:59.817+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:17:59.946 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:17:59.941+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:00.125 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:00.119+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:00.209 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:00.204+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:00.759 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:00.757+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:00.804 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:00.800+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:00.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:00.826+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:00.910 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:00.906+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:01.172 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:01.167+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:01.217 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:01.205+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:01.786 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:01.781+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:01.801 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:01.797+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:01.861 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:01.853+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:01.884 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:01.880+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:02.182 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:02.177+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:02.199 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:02.194+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:02.759 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:02.756+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:02.817 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:02.812+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:02.905 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:02.899+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:02.934 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:02.930+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:03.167 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:03.162+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:03.208 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:03.203+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:03.760 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:03.755+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:03.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:03.767+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:03.870 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:03.866+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:03.935 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:03.930+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:04.192 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:04.187+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:04.195 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:04.190+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:04.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:04.775+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:04.793 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:04.789+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:04.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:04.827+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:04.931 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:04.927+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:05.189 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:05.184+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:05.200 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:05.195+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:05.745 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:05.740+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:05.831 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:05.827+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:05.831 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:05.827+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:05.913 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:05.908+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:06.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:06.160+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:06.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:06.217+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:06.723 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:06.719+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:06.857 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:06.853+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:06.868 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:06.864+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:06.874 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:06.869+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:07.184 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:07.179+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:07.204 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:07.200+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:07.719 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:07.714+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:07.817 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:07.813+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:07.841 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:07.836+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:07.864 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:07.860+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:08.177 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:08.172+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:08.184 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:08.179+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:08.750 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:08.747+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:08.788 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:08.785+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:08.826 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:08.822+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:08.872 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:08.867+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:09.152 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:09.147+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:09.177 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:09.172+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:09.771 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:09.768+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:09.832 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:09.829+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:09.851 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:09.848+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:09.877 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:09.873+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:10.172 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:10.167+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:10.197 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:10.192+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:10.746 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:10.743+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:10.830 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:10.826+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:10.852 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:10.848+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:10.874 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:10.870+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:11.225 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:11.221+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:11.253 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:11.241+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:11.755 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:11.751+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:11.839 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:11.835+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:11.842 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:11.839+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:11.900 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:11.897+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:12.220 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:12.217+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:12.261 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:12.256+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:12.726 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:12.723+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:12.825 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:12.820+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:12.869 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:12.866+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:12.914 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:12.910+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:13.238 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:13.233+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:13.282 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:13.277+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:13.772 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:13.769+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:13.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:13.783+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:13.874 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:13.871+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:13.934 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:13.929+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:14.255 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:14.249+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:14.304 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:14.299+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:14.735 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:14.730+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:14.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:14.798+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:14.881 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:14.878+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:14.888 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:14.884+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:15.267 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:15.263+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:15.307 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:15.302+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:15.787 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:15.783+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:15.852 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:15.847+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:15.886 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:15.882+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:15.887 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:15.883+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:16.277 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:16.270+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:16.357 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:16.345+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:16.768 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:18:16.764+0000 1694f700 -1 osd.2 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.084363+0000 front 2019-12-09T20:17:08.074306+0000 (oldest deadline 2019-12-09T20:17:53.856540+0000) 2019-12-09T20:18:16.840 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:18:16.836+0000 1694f700 -1 osd.5 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:06.995397+0000 front 2019-12-09T20:17:07.328397+0000 (oldest deadline 2019-12-09T20:17:51.071827+0000) 2019-12-09T20:18:16.886 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:18:16.881+0000 1694f700 -1 osd.1 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.457658+0000 front 2019-12-09T20:17:08.458383+0000 (oldest deadline 2019-12-09T20:17:53.123590+0000) 2019-12-09T20:18:16.916 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:16.912+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:17.345 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:18:17.336+0000 1694f700 -1 osd.6 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:07.484237+0000 front 2019-12-09T20:17:07.548565+0000 (oldest deadline 2019-12-09T20:17:51.936733+0000) 2019-12-09T20:18:17.397 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:18:17.380+0000 1694f700 -1 osd.7 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:09.308147+0000 front 2019-12-09T20:17:09.417549+0000 (oldest deadline 2019-12-09T20:17:50.996857+0000) 2019-12-09T20:18:17.924 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:18:17.909+0000 1694f700 -1 osd.0 496 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:17:08.900770+0000 front 2019-12-09T20:17:08.905268+0000 (oldest deadline 2019-12-09T20:17:51.508770+0000) 2019-12-09T20:18:20.889 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:57:12.740 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.905 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:57:12.730 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.907 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:57:12.757 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.913 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:57:12.738 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.917 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:57:12.742 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.921 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:57:12.746 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:20.994 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:08:57:12.819 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:21.034 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:08:57:12.885 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:18:21.041 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:08:57:12.865 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:19:24.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:19:24.264+0000 31f87700 -1 osd.5 500 build_incremental_map_msg missing incremental map 500 2019-12-09T20:19:24.618 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:19:24.599+0000 38794700 -1 osd.5 500 build_incremental_map_msg missing incremental map 500 2019-12-09T20:19:24.809 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:19:24.776+0000 37f93700 -1 osd.5 500 build_incremental_map_msg missing incremental map 500 2019-12-09T20:19:27.683 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:27.669+0000 30f85700 -1 osd.3 502 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:27.821 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:27.768+0000 31786700 -1 osd.3 502 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:27.877 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:27.806+0000 3378a700 -1 osd.3 502 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:28.089 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:27.947+0000 3578e700 -1 osd.3 502 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:28.120 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:27.957+0000 3478c700 -1 osd.3 502 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:29.393 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:29.316+0000 36f91700 -1 osd.3 503 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:31.046 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:31.033+0000 38794700 -1 osd.3 504 build_incremental_map_msg missing incremental map 504 2019-12-09T20:19:31.247 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:31.208+0000 114ae700 -1 osd.3 504 build_incremental_map_msg missing incremental map 504 2019-12-09T20:19:31.471 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:31.462+0000 36f91700 -1 osd.3 504 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:31.473 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:31.462+0000 36f91700 -1 osd.3 504 build_incremental_map_msg missing incremental map 504 2019-12-09T20:19:31.474 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:31.465+0000 36f91700 -1 osd.3 504 build_incremental_map_msg missing incremental map 504 2019-12-09T20:19:32.993 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:32.985+0000 3478c700 -1 osd.3 504 build_incremental_map_msg missing incremental map 502 2019-12-09T20:19:32.993 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:19:32.986+0000 3478c700 -1 osd.3 504 build_incremental_map_msg missing incremental map 504 2019-12-09T20:20:32.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:31.998+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:32.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:32.444+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:33.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:33.034+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:33.495 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:33.491+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:33.966 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:33.961+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:34.037 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:34.032+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:34.458 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:34.453+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:34.537 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:34.533+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:34.988 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:34.984+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:35.004 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:34.999+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:35.062 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:35.058+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:35.068 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:35.062+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:35.509 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:35.503+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:35.525 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:35.520+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:35.962 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:35.958+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:36.017 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:36.012+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:36.032 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:36.027+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:36.052 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:36.047+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:36.517 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:36.512+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:36.517 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:36.511+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:37.002 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:36.997+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:37.046 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:37.042+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:37.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:37.056+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:37.063 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:37.059+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:37.543 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:37.540+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:37.548 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:37.542+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:37.956 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:37.952+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:38.028 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:38.024+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:38.044 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:38.040+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:38.091 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:38.086+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:38.513 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:38.503+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:38.543 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:38.539+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:38.929 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:38.925+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:39.015 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:39.010+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:39.068 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:39.063+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:39.098 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:39.094+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:39.500 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:20:39.496+0000 1694f700 -1 osd.2 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:46.623758+0000 front 2019-12-09T20:19:46.622782+0000 (oldest deadline 2019-12-09T20:20:32.240598+0000) 2019-12-09T20:20:39.540 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:39.535+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:39.968 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:39.963+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:39.997 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:39.992+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:40.062 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:40.056+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:40.110 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:40.105+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:40.502 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:40.497+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:40.961 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:40.956+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:41.035 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:41.030+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:41.042 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:41.037+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:41.141 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:41.134+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:41.528 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:41.523+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:41.935 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:41.931+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:42.004 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:41.999+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:42.041 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:42.036+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:42.114 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:42.110+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:42.577 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:42.573+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:42.928 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:42.923+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:42.968 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:42.964+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:43.090 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:43.085+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:43.165 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:43.161+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:43.552 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:43.547+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:43.972 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:43.969+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:44.007 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:44.001+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:44.109 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:44.104+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:44.208 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:44.204+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:44.513 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:44.509+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:44.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:44.994+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:45.002 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:44.997+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:45.102 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:45.097+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:45.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:45.232+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:45.481 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:45.476+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:45.952 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:45.948+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:46.042 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:46.037+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:46.131 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:46.126+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:46.227 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:46.223+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:46.528 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:46.523+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:46.988 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:46.984+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:47.078 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:47.073+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:47.120 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:47.115+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:47.193 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:47.189+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:47.579 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:47.573+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:48.031 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:48.027+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:48.100 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:48.095+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:48.153 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:48.148+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:48.219 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:48.215+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:48.612 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:48.607+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:49.060 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:49.057+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:49.133 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:49.129+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:49.169 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:49.165+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:49.258 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:49.255+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:49.591 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:49.587+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:50.042 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:50.038+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:50.117 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:50.113+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:50.202 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:50.196+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:50.299 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:50.294+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:50.638 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:50.633+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:51.045 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:51.041+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:51.154 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:51.150+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:51.173 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:51.168+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:51.305 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:51.301+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:51.653 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:51.649+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:52.073 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:52.068+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:52.145 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:52.138+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:52.184 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:52.179+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:52.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:52.311+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:52.616 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:52.611+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:53.089 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:53.085+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:53.121 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:53.112+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:53.162 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:53.153+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:53.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:53.337+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:53.606 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:53.602+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:54.091 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:54.087+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:54.127 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:54.122+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:54.198 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:54.194+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:54.334 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:54.331+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:54.657 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:54.653+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:55.098 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:55.093+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:55.104 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:55.101+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:55.242 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:55.237+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:55.290 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:55.286+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:55.713 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:55.705+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:56.117 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:56.113+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:56.120 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:56.116+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:56.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:56.250+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:56.266 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:56.262+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:56.761 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:56.756+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:57.114 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:57.109+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:57.177 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:57.172+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:57.307 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:57.297+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:57.309 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:57.302+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:57.753 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:57.748+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:58.152 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:58.144+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:58.203 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:58.199+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:58.287 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:58.283+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:58.317 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:58.311+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:58.780 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:58.775+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:20:59.127 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:20:59.122+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:20:59.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:20:59.236+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:20:59.274 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:20:59.269+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:20:59.340 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:20:59.335+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:20:59.792 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:20:59.787+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:00.172 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:00.167+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:00.220 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:00.214+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:00.302 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:00.298+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:00.305 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:00.299+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:00.771 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:00.767+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:01.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:01.176+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:01.205 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:01.199+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:01.289 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:01.284+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:01.344 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:01.338+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:01.732 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:01.728+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:02.137 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:02.133+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:02.168 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:02.162+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:02.310 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:02.304+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:02.330 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:02.326+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:02.731 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:02.727+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:03.104 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:03.100+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:03.133 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:03.128+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:03.286 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:03.282+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:03.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:03.319+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:03.738 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:03.734+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:04.077 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:04.070+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:04.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:04.105+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:04.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:04.300+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:04.316 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:04.311+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:04.717 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:04.712+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:05.039 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:05.033+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:05.162 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:05.157+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:05.295 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:05.290+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:05.321 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:05.316+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:05.679 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:05.674+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:06.027 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:06.023+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:06.125 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:06.120+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:06.276 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:06.271+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:06.319 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:06.314+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:06.695 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:06.690+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:07.051 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:07.046+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:07.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:07.081+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:07.249 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:07.244+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:07.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:07.292+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:21:07.737 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:21:07.732+0000 1694f700 -1 osd.6 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.224984+0000 front 2019-12-09T20:19:49.408603+0000 (oldest deadline 2019-12-09T20:20:33.738420+0000) 2019-12-09T20:21:08.048 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:21:08.043+0000 1694f700 -1 osd.1 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:47.595908+0000 front 2019-12-09T20:19:47.817801+0000 (oldest deadline 2019-12-09T20:20:31.519801+0000) 2019-12-09T20:21:08.071 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:21:08.066+0000 1694f700 -1 osd.5 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:49.959009+0000 front 2019-12-09T20:19:49.526458+0000 (oldest deadline 2019-12-09T20:20:33.536649+0000) 2019-12-09T20:21:08.139 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:21:08.133+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e515 v515) 2019-12-09T20:21:08.280 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:21:08.249+0000 1694f700 -1 osd.7 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.336498+0000 front 2019-12-09T20:19:48.329788+0000 (oldest deadline 2019-12-09T20:20:34.156538+0000) 2019-12-09T20:21:08.289 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:21:08.284+0000 1694f700 -1 osd.0 515 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:19:48.455992+0000 front 2019-12-09T20:19:48.951444+0000 (oldest deadline 2019-12-09T20:20:34.294321+0000) 2019-12-09T20:22:13.197 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:01:05.048 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.206 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:01:05.056 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.215 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:01:05.040 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.217 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:01:05.042 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.220 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:01:05.045 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.221 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:01:05.046 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.486 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:01:05.336 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.546 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:01:05.371 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:22:13.552 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:01:05.377 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:23:11.200 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:11.188+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:11.358 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:11.354+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:11.836 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:11.832+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:12.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:12.073+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:12.198 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:12.194+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:12.389 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:12.385+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:12.813 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:12.810+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:13.114 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:13.110+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:13.191 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:13.185+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:13.379 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:13.374+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:13.787 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:13.783+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:14.120 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:14.116+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:14.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:14.151+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:14.380 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:14.376+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:14.780 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:14.777+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:14.956 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:14.951+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:15.073 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:15.070+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:15.211 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:15.206+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:15.215 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:15.211+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:15.405 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:15.402+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:15.738 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:15.734+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:16.009 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:16.004+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:16.082 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:16.079+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:16.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:16.218+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:16.254 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:16.250+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:16.396 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:16.393+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:16.705 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:16.702+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:17.060 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:17.055+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:17.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:17.124+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:17.191 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:17.186+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:17.251 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:17.246+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:17.409 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:17.405+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:17.736 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:17.732+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:18.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:18.074+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:18.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:18.150+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:18.180 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:18.176+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:18.211 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:18.207+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:18.368 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:18.364+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:18.790 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:18.786+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:19.068 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:19.063+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:19.153 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:19.149+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:19.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:19.174+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:19.212 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:19.208+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:19.356 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:19.352+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:19.818 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:19.815+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:20.105 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:20.101+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:20.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:20.177+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:20.206 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:20.200+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:20.207 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:20.202+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:20.330 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:20.327+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:20.857 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:20.854+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:21.118 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:21.113+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:21.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:21.192+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:21.246 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:21.240+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:21.256 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:21.251+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:21.304 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:21.300+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:21.880 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:21.875+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:22.152 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:22.147+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:22.223 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:22.219+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:22.241 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:22.237+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:22.284 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:22.279+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:22.308 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:22.304+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:22.855 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:22.849+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:23.129 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:23.124+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:23.204 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:23.197+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:23.274 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:23.270+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:23.295 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:23.290+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:23.339 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:23.335+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:23.829 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:23.826+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:24.121 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:24.116+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:24.157 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:24.153+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:24.273 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:24.268+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:24.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:24.284+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:24.298 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:24.295+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:24.876 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:24.871+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:25.131 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:25.125+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:25.182 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:25.177+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:25.261 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:25.257+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:25.293 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:25.289+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:25.327 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:25.324+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:25.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:25.918+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:26.142 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:26.138+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:26.204 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:26.199+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:26.239 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:26.233+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:26.266 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:26.260+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:26.377 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:26.372+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:26.885 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:26.880+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:27.149 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:27.143+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:27.198 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:27.193+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:27.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:27.283+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:27.294 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:27.287+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:27.359 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:27.354+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:27.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:27.897+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:28.107 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:28.102+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:28.213 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:28.208+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:28.272 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:28.268+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:28.298 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:28.294+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:28.370 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:28.366+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:28.920 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:28.914+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:29.138 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:29.132+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:29.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:29.236+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:29.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:29.282+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:29.298 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:29.293+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:29.409 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:29.405+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:29.890 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:29.883+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:30.159 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:30.154+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:30.210 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:30.206+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:30.291 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:30.287+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:30.327 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:30.322+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:30.442 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:30.438+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:30.891 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:30.887+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:31.129 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:31.124+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:31.208 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:31.203+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:31.264 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:31.259+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:31.369 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:31.356+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:31.484 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:31.479+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:31.891 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:31.887+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:32.151 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:32.146+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:32.188 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:32.183+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:32.228 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:32.223+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:32.356 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:32.351+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:32.478 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:32.472+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:32.869 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:32.863+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:33.124 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:33.120+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:33.231 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:33.222+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:33.277 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:33.272+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:33.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:33.313+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:33.465 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:33.460+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:33.896 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:33.890+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:34.095 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:34.090+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:34.271 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:34.267+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:34.275 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:34.272+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:34.346 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:34.339+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:34.431 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:34.426+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:34.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:34.942+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:35.056 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:35.048+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:35.239 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:35.234+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:35.260 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:35.255+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:35.347 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:35.342+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:35.425 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:35.419+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:35.987 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:35.982+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:36.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:36.055+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:36.208 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:36.203+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:36.286 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:36.281+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:36.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:36.380+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:36.394 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:36.389+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:36.996 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:36.991+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:37.033 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:37.029+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:37.172 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:37.167+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:37.327 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:37.322+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:37.360 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:37.354+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:37.397 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:37.392+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:37.989 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:37.984+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:38.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:38.074+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:38.171 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:38.166+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:38.362 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:38.357+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:38.396 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:38.390+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:38.397 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:38.391+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:38.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:38.942+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:39.072 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:39.068+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:39.141 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:39.136+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:39.397 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:39.392+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:39.448 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:39.442+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:39.448 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:39.443+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:39.901 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:39.896+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:40.072 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:40.067+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:40.134 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:40.130+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:40.401 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:40.396+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:40.417 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:40.408+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:40.445 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:40.441+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:40.890 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:40.886+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:41.037 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:41.028+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:41.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:41.161+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:41.419 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:41.414+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:41.442 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:41.437+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:41.458 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:41.453+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:41.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:41.882+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:41.991 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:41.986+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:42.204 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:42.199+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:42.379 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:42.373+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:42.473 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:42.468+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:42.487 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:42.483+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:42.842 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:42.838+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:42.960 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:42.955+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:43.201 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:43.197+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:43.376 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:43.371+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:43.453 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:43.449+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:43.516 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:43.512+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:43.824 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:43.819+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:44.013 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:44.008+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:44.174 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:44.168+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:44.248 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:23:44.244+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e522 v522) 2019-12-09T20:23:44.387 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:44.382+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:44.481 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:44.476+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:44.486 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:44.481+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:44.867 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:44.860+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:44.982 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:44.977+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:45.176 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:45.171+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:45.373 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:45.367+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:45.509 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:45.503+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:45.531 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:45.527+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:45.847 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:45.842+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:46.035 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:46.030+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:46.223 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:46.218+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:46.421 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:46.417+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:46.477 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:46.465+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:46.581 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:46.576+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:46.840 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:46.832+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:47.011 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:47.008+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:47.241 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:47.236+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:47.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:47.380+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:47.430 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:47.424+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:47.633 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:47.626+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:47.823 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:47.819+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:48.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:48.011+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:48.237 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:48.233+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:48.360 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:48.355+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:48.419 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:48.414+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:48.585 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:48.580+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:48.858 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:48.853+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:49.016 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:49.005+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:49.239 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:49.235+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:49.263 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:23:49.259+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e522 v522) 2019-12-09T20:23:49.404 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:49.400+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:49.466 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:49.461+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:49.547 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:49.542+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:49.822 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:49.817+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:50.032 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:50.026+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:50.222 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:50.216+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:50.401 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:50.396+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:50.422 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:50.417+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:50.543 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:50.538+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:50.793 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:50.788+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:51.084 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:51.068+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:51.187 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:51.181+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:51.390 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:51.385+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:51.406 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:51.401+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:51.523 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:51.519+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:51.811 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:51.807+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:52.067 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:52.063+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:52.142 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:52.136+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:52.394 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:52.390+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:52.400 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:52.396+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:52.519 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:52.514+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:52.834 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:52.829+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:53.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:53.058+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:53.163 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:53.160+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:53.375 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:53.369+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:53.441 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:53.437+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:23:53.548 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:23:53.538+0000 1694f700 -1 osd.5 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.212688+0000 front 2019-12-09T20:22:29.714685+0000 (oldest deadline 2019-12-09T20:23:14.473210+0000) 2019-12-09T20:23:53.876 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:23:53.872+0000 1694f700 -1 osd.2 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:26.800344+0000 front 2019-12-09T20:22:26.737836+0000 (oldest deadline 2019-12-09T20:23:11.788039+0000) 2019-12-09T20:23:54.028 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:23:54.023+0000 1694f700 -1 osd.7 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.701966+0000 front 2019-12-09T20:22:29.333547+0000 (oldest deadline 2019-12-09T20:23:14.007347+0000) 2019-12-09T20:23:54.209 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:23:54.205+0000 1694f700 -1 osd.6 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.091158+0000 front 2019-12-09T20:22:27.092413+0000 (oldest deadline 2019-12-09T20:23:10.414133+0000) 2019-12-09T20:23:54.341 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:23:54.338+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e522 v522) 2019-12-09T20:23:54.368 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:23:54.364+0000 1694f700 -1 osd.1 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:29.035493+0000 front 2019-12-09T20:22:29.121358+0000 (oldest deadline 2019-12-09T20:23:11.320118+0000) 2019-12-09T20:23:54.468 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:23:54.463+0000 1694f700 -1 osd.0 522 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:22:27.522272+0000 front 2019-12-09T20:22:27.621808+0000 (oldest deadline 2019-12-09T20:23:10.397015+0000) 2019-12-09T20:24:37.121 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:24:37.080+0000 32f89700 -1 osd.0 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.286 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:24:37.117+0000 32b88700 -1 osd.1 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.325 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:24:37.149+0000 3478c700 -1 osd.0 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.408 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:24:37.217+0000 37391700 -1 osd.1 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.439 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:24:37.286+0000 33f8b700 -1 osd.0 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.475 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:24:37.285+0000 3438b700 -1 osd.1 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:24:37.560 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:24:37.497+0000 11caf700 -1 osd.1 526 build_incremental_map_msg missing incremental map 526 2019-12-09T20:25:41.342 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:41.338+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:42.397 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:42.388+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:42.655 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:42.651+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:42.676 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:42.673+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:42.900 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:42.896+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:43.069 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:43.065+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:43.363 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:43.358+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:43.637 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:43.633+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:43.716 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:43.712+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:43.813 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:43.808+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:43.942 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:43.938+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:44.057 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:44.053+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:44.364 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:44.360+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:44.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:44.622+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:44.748 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:44.743+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:44.791 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:44.786+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:44.970 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:44.965+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:45.100 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:45.096+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:45.391 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:45.386+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:45.674 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:45.670+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:45.725 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:45.720+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:45.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:45.805+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:45.975 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:45.970+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:46.106 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:46.101+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:46.348 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:46.343+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:46.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:46.667+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:46.768 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:46.764+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:46.822 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:46.817+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:47.018 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:47.011+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:47.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:47.105+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:47.336 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:47.332+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:47.644 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:47.639+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:47.782 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:47.779+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:47.850 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:47.844+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:47.973 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:47.968+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:48.124 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:48.120+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:48.327 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:48.322+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:48.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:48.608+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:48.825 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:48.821+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:48.885 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:48.881+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:49.008 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:49.004+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:49.125 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:49.115+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:49.354 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:49.350+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:49.663 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:49.659+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:49.833 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:49.824+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:49.905 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:49.901+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:50.003 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:49.999+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:50.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:50.082+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:50.314 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:50.309+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:50.618 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:50.614+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:50.850 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:50.846+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:50.888 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:50.883+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:51.004 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:51.000+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:51.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:51.129+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:51.310 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:51.306+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:51.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:51.638+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:51.873 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:51.869+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:51.890 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:51.886+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:51.994 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:51.989+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:52.118 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:52.114+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:52.308 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:52.304+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:52.628 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:52.624+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:52.922 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:52.918+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:52.923 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:52.918+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:53.009 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:53.005+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:53.153 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:53.150+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:53.361 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:53.356+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:53.626 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:53.623+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:53.900 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:53.893+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:53.931 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:53.924+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:54.061 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:54.056+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:54.159 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:54.155+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:54.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:54.384+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:54.657 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:54.653+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:54.924 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:54.917+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:54.939 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:54.936+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:55.113 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:55.108+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:55.118 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:55.115+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:55.357 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:55.352+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:55.650 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:55.646+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:55.950 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:55.945+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:55.952 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:55.947+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:56.092 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:56.087+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:56.157 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:56.152+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:56.387 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:56.381+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:56.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:56.638+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:56.946 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:56.942+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:56.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:56.981+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:57.088 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:57.084+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:57.142 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:57.139+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:57.350 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:57.346+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:57.626 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:57.622+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:57.918 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:57.913+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:57.952 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:57.948+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:58.102 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:58.098+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:58.160 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:58.154+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:58.379 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:58.375+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:58.620 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:58.616+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:58.883 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:58.879+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:58.990 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:58.979+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:25:59.121 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:25:59.117+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:25:59.203 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:25:59.198+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:25:59.409 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:25:59.400+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:25:59.635 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:25:59.630+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:25:59.886 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:25:59.881+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:25:59.973 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:25:59.968+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:00.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:00.118+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:00.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:00.197+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:00.386 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:00.381+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:00.659 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:00.655+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:00.875 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:00.872+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:00.933 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:00.928+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:01.160 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:01.157+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:01.173 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:01.168+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:01.429 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:01.425+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:01.657 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:01.652+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:01.880 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:01.875+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:01.888 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:01.883+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:02.190 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:02.187+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:02.218 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:02.214+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:02.456 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:02.453+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:02.617 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:02.612+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:02.842 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:02.839+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:02.927 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:02.922+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:03.202 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:03.198+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:03.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:03.261+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:03.416 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:03.411+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:03.650 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:03.645+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:03.866 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:03.862+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:03.945 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:03.940+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:04.159 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:04.154+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:04.298 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:04.293+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:04.449 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:04.444+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:04.684 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:04.679+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:04.902 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:04.898+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:04.950 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:04.945+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:05.210 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:05.205+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:05.295 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:05.290+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:05.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:05.458+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:05.718 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:05.714+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:05.932 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:05.927+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:05.947 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:05.943+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:06.189 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:06.183+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:06.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:06.303+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:06.473 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:06.468+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:06.707 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:06.700+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:06.944 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:06.939+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:06.965 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:06.960+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:07.161 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:07.156+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:07.288 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:07.283+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:07.475 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:07.469+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:07.730 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:07.725+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:07.922 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:07.917+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:07.975 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:07.970+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:08.152 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:08.144+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:08.311 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:08.306+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:08.445 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:08.440+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:08.769 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:08.763+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:08.945 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:08.941+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:08.964 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:08.958+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:09.144 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:09.139+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:09.315 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:09.310+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:09.482 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:09.477+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:09.802 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:09.797+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:09.901 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:09.895+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:09.983 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:09.978+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:10.126 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:10.121+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:10.182 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:02.030 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.182 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:02.032 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.259 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:02.084 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.261 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:02.086 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.263 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:02.088 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.264 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:02.089 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.351 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:10.347+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:10.358 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:02.209 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.439 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:02.264 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.505 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:10.501+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:10.511 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:02.336 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:10.831 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:10.826+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:10.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:10.886+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:11.033 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:11.028+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:11.183 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:11.174+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:11.324 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:11.319+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:11.525 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:11.522+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:11.855 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:11.852+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:11.858 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:11.854+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:12.086 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:12.081+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:12.188 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:12.184+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:12.336 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:12.331+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:12.525 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:12.521+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:12.826 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:12.820+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:12.865 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:12.855+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:13.092 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:13.086+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:13.194 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:13.190+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:13.317 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:13.311+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:13.562 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:13.557+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:13.780 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:13.775+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:13.828 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:13.823+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:14.082 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:14.077+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:14.208 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:14.204+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:14.276 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:14.271+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:14.552 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:14.544+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:14.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:14.775+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:14.790 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:14.785+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:15.073 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:15.062+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:15.092 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:26:15.089+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e529 v529) 2019-12-09T20:26:15.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:15.205+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:15.258 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:15.254+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:15.548 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:15.543+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:15.745 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:15.741+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:15.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:15.774+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:16.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:16.026+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:16.234 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:16.230+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:16.289 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:16.284+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:16.558 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:16.553+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:16.715 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:16.711+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:16.829 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:16.824+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:17.049 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:17.044+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:17.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:17.217+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:17.309 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:17.305+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:17.606 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:17.601+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:17.763 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:17.758+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:17.792 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:17.786+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:18.039 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:18.033+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:18.274 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:18.269+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:18.310 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:18.304+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:18.633 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:18.621+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:18.749 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:18.744+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:18.804 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:18.799+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:19.001 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:18.996+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:19.237 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:19.231+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:19.302 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:19.297+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:19.647 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:19.643+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:19.739 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:19.735+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:19.853 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:19.848+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:19.998 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:19.993+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:20.111 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:26:20.106+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e529 v529) 2019-12-09T20:26:20.250 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:20.245+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:20.316 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:20.311+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:20.617 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:20.610+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:20.704 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:20.699+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:20.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:20.828+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:21.009 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:21.003+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:21.206 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:21.203+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:21.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:21.304+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:21.661 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:21.657+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:21.661 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:21.657+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:21.841 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:21.836+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:22.011 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:22.007+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:22.187 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:22.182+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:22.324 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:22.320+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:22.628 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:22.623+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:22.669 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:22.664+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:22.848 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:22.839+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:23.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:23.029+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:23.213 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:23.208+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:23.352 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:23.348+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:23.656 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:23.651+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:23.692 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:23.688+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:23.833 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:23.829+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:24.017 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:24.013+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:24.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:24.204+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:24.392 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:26:24.388+0000 1694f700 -1 osd.1 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.402216+0000 front 2019-12-09T20:24:59.407838+0000 (oldest deadline 2019-12-09T20:25:42.845847+0000) 2019-12-09T20:26:24.622 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:26:24.617+0000 1694f700 -1 osd.5 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:59.063929+0000 front 2019-12-09T20:25:00.694789+0000 (oldest deadline 2019-12-09T20:25:43.449004+0000) 2019-12-09T20:26:24.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:26:24.723+0000 1694f700 -1 osd.6 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:55.132436+0000 front 2019-12-09T20:25:00.704756+0000 (oldest deadline 2019-12-09T20:25:40.670744+0000) 2019-12-09T20:26:24.874 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:26:24.870+0000 1694f700 -1 osd.2 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.519133+0000 front 2019-12-09T20:24:57.530324+0000 (oldest deadline 2019-12-09T20:25:42.175886+0000) 2019-12-09T20:26:24.993 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:26:24.987+0000 1694f700 -1 osd.0 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:25:00.247149+0000 front 2019-12-09T20:25:00.246747+0000 (oldest deadline 2019-12-09T20:25:42.517077+0000) 2019-12-09T20:26:25.201 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:26:25.196+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e529 v529) 2019-12-09T20:26:25.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:26:25.220+0000 1694f700 -1 osd.7 529 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:24:57.428737+0000 front 2019-12-09T20:24:57.430497+0000 (oldest deadline 2019-12-09T20:25:42.080534+0000) 2019-12-09T20:26:30.973 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:22.821 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:30.980 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:22.831 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:30.993 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:22.813 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.004 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:22.819 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.004 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:22.820 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.005 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:22.824 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.049 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:05:22.871 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.068 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:05:22.918 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:26:31.070 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:05:22.895 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:27:07.871 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:27:07.615+0000 36f91700 -1 osd.0 533 build_incremental_map_msg missing incremental map 533 2019-12-09T20:27:07.939 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:27:07.795+0000 3578e700 -1 osd.0 533 build_incremental_map_msg missing incremental map 533 2019-12-09T20:27:07.965 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:27:07.863+0000 37f93700 -1 osd.0 533 build_incremental_map_msg missing incremental map 533 2019-12-09T20:27:08.041 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:27:07.961+0000 3478c700 -1 osd.0 533 build_incremental_map_msg missing incremental map 533 2019-12-09T20:27:10.389 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:10.345+0000 3578e700 -1 osd.3 534 build_incremental_map_msg missing incremental map 534 2019-12-09T20:27:10.517 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:10.461+0000 31786700 -1 osd.3 534 build_incremental_map_msg missing incremental map 534 2019-12-09T20:27:10.584 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:10.524+0000 30f85700 -1 osd.3 534 build_incremental_map_msg missing incremental map 534 2019-12-09T20:27:11.042 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:10.924+0000 31786700 -1 osd.3 534 build_incremental_map_msg missing incremental map 534 2019-12-09T20:27:12.586 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:12.566+0000 33f8b700 -1 osd.3 534 build_incremental_map_msg missing incremental map 534 2019-12-09T20:27:14.454 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:27:14.435+0000 36f91700 -1 osd.3 535 build_incremental_map_msg missing incremental map 534 2019-12-09T20:28:12.302 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:12.297+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:12.695 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:12.690+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:13.257 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:13.253+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:13.626 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:13.623+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:13.721 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:13.716+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:14.219 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:14.214+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:14.646 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:14.642+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:14.696 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:14.691+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:15.208 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:15.204+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:15.629 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:15.625+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:15.676 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:15.670+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:16.202 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:16.197+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:16.216 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:16.212+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:16.421 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:16.416+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:16.607 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:16.602+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:16.627 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:16.622+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:16.641 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:16.637+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:17.230 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:17.226+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:17.249 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:17.245+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:17.456 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:17.453+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:17.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:17.560+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:17.585 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:17.581+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:17.656 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:17.638+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:18.212 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:18.207+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:18.224 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:18.220+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:18.436 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:18.433+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:18.546 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:18.543+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:18.626 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:18.621+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:18.634 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:18.629+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:19.193 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:19.189+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:19.270 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:19.266+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:19.420 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:19.417+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:19.541 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:19.538+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:19.652 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:19.647+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:19.687 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:19.682+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:20.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:20.181+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:20.230 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:20.227+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:20.424 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:20.419+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:20.583 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:20.579+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:20.688 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:20.684+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:20.695 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:20.690+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:21.153 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:21.149+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:21.251 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:21.247+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:21.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:21.394+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:21.596 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:21.592+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:21.646 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:21.641+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:21.733 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:21.730+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:22.133 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:22.129+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:22.209 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:22.205+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:22.415 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:22.410+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:22.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:22.562+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:22.615 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:22.608+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:22.776 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:22.772+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:23.171 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:23.167+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:23.186 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:23.178+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:23.372 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:23.368+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:23.551 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:23.547+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:23.578 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:23.573+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:23.780 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:23.769+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:24.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:24.160+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:24.196 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:24.191+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:24.366 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:24.362+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:24.549 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:24.544+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:24.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:24.590+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:24.767 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:24.762+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:25.186 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:25.179+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:25.194 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:25.188+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:25.383 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:25.380+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:25.537 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:25.532+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:25.643 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:25.639+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:25.781 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:25.777+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:26.206 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:26.200+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:26.218 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:26.213+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:26.424 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:26.418+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:26.555 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:26.550+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:26.674 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:26.670+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:26.816 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:26.809+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:27.216 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:27.210+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:27.223 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:27.218+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:27.465 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:27.461+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:27.602 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:27.593+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:27.644 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:27.639+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:27.813 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:27.808+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:28.173 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:28.168+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:28.247 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:28.242+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:28.512 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:28.507+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:28.603 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:28.599+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:28.698 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:28.693+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:28.863 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:28.859+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:29.201 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:29.196+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:29.207 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:29.203+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:29.516 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:29.512+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:29.614 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:29.609+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:29.684 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:29.678+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:29.870 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:29.866+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:30.179 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:30.175+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:30.182 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:30.178+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:30.507 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:30.502+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:30.612 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:30.607+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:30.650 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:30.645+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:30.838 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:30.834+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:31.165 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:31.160+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:31.206 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:31.201+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:31.558 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:31.553+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:31.611 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:31.606+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:31.637 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:31.631+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:31.798 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:31.793+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:32.128 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:32.122+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:32.221 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:32.216+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:32.554 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:32.548+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:32.618 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:32.612+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:32.657 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:32.653+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:32.804 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:32.800+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:33.095 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:33.091+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:33.268 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:33.264+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:33.551 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:33.545+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:33.632 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:33.627+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:33.673 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:33.668+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:33.806 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:33.801+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:34.136 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:34.132+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:34.313 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:34.308+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:34.535 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:34.530+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:34.614 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:34.610+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:34.668 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:34.663+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:34.811 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:34.807+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:35.150 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:35.141+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:35.303 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:35.299+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:35.523 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:35.518+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:35.640 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:35.635+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:35.712 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:35.705+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:35.803 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:35.796+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:36.133 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:36.128+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:36.260 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:36.256+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:36.508 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:36.503+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:36.595 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:36.589+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:36.689 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:36.684+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:36.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:36.790+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:37.129 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:37.125+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:37.244 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:37.238+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:37.554 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:37.549+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:37.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:37.555+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:37.712 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:37.706+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:37.752 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:37.747+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:38.113 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:38.108+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:38.208 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:38.203+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:38.526 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:38.520+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:38.557 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:38.552+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:38.734 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:38.730+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:38.772 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:38.768+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:39.136 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:39.132+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:39.251 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:39.245+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:39.504 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:39.499+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:39.574 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:39.569+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:39.732 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:39.727+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:39.733 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:39.726+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:40.095 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:40.091+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:40.292 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:40.288+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:40.548 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:40.544+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:40.577 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:40.573+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:40.692 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:40.688+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:40.747 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:40.742+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:41.093 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:41.089+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:41.342 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:41.337+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:41.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:41.529+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:41.583 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:41.577+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:41.708 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:41.703+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:41.751 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:41.746+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:42.129 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:42.125+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:42.331 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:42.326+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:42.532 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:42.527+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:42.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:42.591+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:42.712 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:42.707+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:42.755 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:42.751+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:43.155 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:43.150+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:43.361 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:43.356+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:43.493 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:43.487+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:43.551 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:43.545+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:43.711 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:43.706+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:43.776 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:43.771+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:44.115 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:44.110+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:44.404 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:44.399+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:44.524 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:44.520+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:44.576 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:44.571+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:44.725 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:44.720+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:44.765 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:44.760+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:45.075 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:45.070+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:45.425 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:45.421+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:45.542 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:45.538+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:45.550 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:45.545+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:45.696 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:45.690+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:45.742 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:45.738+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:45.784 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:28:45.781+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e536 v536) 2019-12-09T20:28:46.073 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:46.068+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:46.406 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:46.400+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:46.511 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:46.506+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:46.533 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:46.527+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:46.683 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:46.678+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:46.794 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:46.786+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:47.044 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:47.034+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:47.421 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:47.416+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:47.507 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:47.502+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:47.586 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:47.581+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:47.679 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:47.674+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:47.821 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:47.816+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:48.081 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:48.076+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:48.448 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:48.443+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:48.532 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:48.527+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:48.628 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:48.623+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:48.731 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:48.724+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:48.857 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:48.852+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:49.041 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:49.037+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:49.471 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:49.466+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:49.521 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:49.516+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:49.585 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:49.579+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:49.773 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:49.768+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:49.834 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:49.830+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:50.056 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:50.051+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:50.483 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:50.479+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:50.537 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:50.533+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:50.617 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:50.612+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:50.786 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:50.782+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:50.804 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:28:50.801+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e536 v536) 2019-12-09T20:28:50.827 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:50.823+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:51.020 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:51.012+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:51.498 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:51.494+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:51.551 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:51.546+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:51.639 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:51.634+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:51.805 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:51.802+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:51.811 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:51.808+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:52.069 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:52.067+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:52.538 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:52.533+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:52.591 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:52.586+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:52.665 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:52.661+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:52.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:52.793+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:52.810 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:52.807+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:53.059 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:53.056+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:53.518 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:53.514+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:53.592 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:53.588+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:53.701 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:53.697+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:53.832 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:53.828+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:53.839 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:53.834+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:54.041 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:54.036+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:54.475 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:54.470+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:54.610 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:54.606+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:54.660 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:54.656+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:54.867 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:54.863+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:54.880 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:54.871+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:55.069 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:55.065+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:55.484 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:28:55.479+0000 1694f700 -1 osd.2 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.772548+0000 front 2019-12-09T20:27:29.773929+0000 (oldest deadline 2019-12-09T20:28:15.639772+0000) 2019-12-09T20:28:55.595 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:55.591+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:28:55.687 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:28:55.682+0000 1694f700 -1 osd.1 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:28.210508+0000 front 2019-12-09T20:27:28.295884+0000 (oldest deadline 2019-12-09T20:28:13.491935+0000) 2019-12-09T20:28:55.847 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:28:55.842+0000 1694f700 -1 osd.7 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.332582+0000 front 2019-12-09T20:27:31.686316+0000 (oldest deadline 2019-12-09T20:28:12.031386+0000) 2019-12-09T20:28:55.859 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:28:55.855+0000 1694f700 -1 osd.5 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:29.935947+0000 front 2019-12-09T20:27:29.950306+0000 (oldest deadline 2019-12-09T20:28:15.775043+0000) 2019-12-09T20:28:55.897 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:28:55.894+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e536 v536) 2019-12-09T20:28:56.032 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:28:56.021+0000 1694f700 -1 osd.6 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640143+0000 front 2019-12-09T20:27:31.862162+0000 (oldest deadline 2019-12-09T20:28:11.800586+0000) 2019-12-09T20:28:56.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:28:56.594+0000 1694f700 -1 osd.0 536 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:27:31.640364+0000 front 2019-12-09T20:27:31.242175+0000 (oldest deadline 2019-12-09T20:28:16.040076+0000) 2019-12-09T20:29:39.245 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:29:39.227+0000 3578e700 -1 osd.0 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:39.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:29:39.496+0000 36f91700 -1 osd.0 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:39.708 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:29:39.517+0000 31f87700 -1 osd.2 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:39.712 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:29:39.537+0000 38b94700 -1 osd.2 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:39.785 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:29:39.648+0000 33f8b700 -1 osd.0 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:39.827 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:29:39.681+0000 38794700 -1 osd.0 540 build_incremental_map_msg missing incremental map 540 2019-12-09T20:29:51.572 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:08:43.422 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.596 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:08:43.446 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.636 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:08:43.461 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.654 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:08:43.479 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.667 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:08:43.492 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.670 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:08:43.495 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.933 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:08:43.758 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.936 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:08:43.761 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:29:51.965 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:08:43.816 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.608 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:09:31.458 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.615 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:09:31.466 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.621 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:09:31.446 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.624 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:09:31.449 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.627 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:09:31.452 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.633 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:09:31.456 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.684 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:09:31.509 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.717 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:09:31.542 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:39.729 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:09:31.580 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:30:43.092 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:43.087+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:43.445 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:43.440+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:44.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:44.047+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:44.428 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:44.424+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:44.550 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:44.545+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:44.794 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:44.790+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:45.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:45.022+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:45.479 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:45.474+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:45.535 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:45.532+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:45.738 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:45.734+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:45.813 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:45.810+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:46.023 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:46.017+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:46.199 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:46.194+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:46.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:46.454+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:46.547 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:46.543+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:46.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:46.699+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:46.850 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:46.847+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:47.046 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:47.042+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:47.243 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:47.239+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:47.498 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:47.493+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:47.597 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:47.592+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:47.692 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:47.689+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:47.805 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:47.802+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:48.066 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:48.062+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:48.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:48.249+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:48.514 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:48.509+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:48.557 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:48.552+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:48.712 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:48.708+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:48.799 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:48.794+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:49.076 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:49.072+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:49.306 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:49.302+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:49.498 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:49.493+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:49.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:49.600+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:49.719 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:49.714+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:49.764 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:49.758+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:50.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:50.057+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:50.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:50.301+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:50.525 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:50.520+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:50.612 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:50.607+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:50.736 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:50.733+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:50.764 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:50.760+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:51.086 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:51.082+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:51.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:51.279+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:51.572 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:51.564+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:51.638 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:51.633+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:51.719 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:51.711+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:51.771 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:51.765+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:52.078 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:52.074+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:52.311 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:52.307+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:52.584 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:52.579+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:52.685 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:52.680+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:52.719 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:52.715+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:52.798 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:52.794+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:53.047 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:53.042+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:53.350 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:53.345+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:53.630 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:53.625+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:53.691 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:53.687+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:53.758 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:53.755+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:53.764 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:53.761+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:54.089 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:54.083+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:54.385 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:54.380+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:54.677 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:54.673+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:54.683 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:54.679+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:54.742 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:54.737+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:54.793 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:54.788+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:55.083 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:55.079+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:55.378 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:55.374+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:55.665 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:55.661+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:55.680 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:55.675+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:55.710 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:55.705+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:55.840 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:55.837+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:56.110 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:56.107+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:56.374 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:56.370+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:56.647 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:56.642+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:56.664 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:56.659+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:56.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:56.667+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:56.889 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:56.885+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:57.111 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:57.108+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:57.337 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:57.332+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:57.682 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:57.678+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:57.694 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:57.691+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:57.705 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:57.700+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:57.909 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:57.904+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:58.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:58.072+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:58.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:58.361+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:58.698 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:58.693+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:58.717 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:58.714+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:58.728 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:58.725+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:58.899 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:58.893+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:30:59.112 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:30:59.108+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:30:59.360 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:30:59.354+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:30:59.695 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:30:59.691+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:30:59.723 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:30:59.719+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:30:59.749 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:30:59.745+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:30:59.858 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:30:59.854+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:00.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:00.136+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:00.355 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:00.351+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:00.651 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:00.646+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:00.703 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:00.698+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:00.782 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:00.777+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:00.896 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:00.890+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:01.121 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:01.116+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:01.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:01.360+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:01.633 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:01.629+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:01.690 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:01.685+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:01.787 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:01.782+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:01.935 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:01.929+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:02.108 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:02.103+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:02.334 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:02.324+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:02.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:02.679+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:02.740 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:02.734+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:02.808 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:02.801+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:02.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:02.952+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:03.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:03.125+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:03.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:03.280+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:03.683 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:03.679+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:03.743 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:03.738+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:03.805 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:03.801+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:03.956 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:03.951+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:04.096 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:04.092+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:04.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:04.250+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:04.676 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:04.672+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:04.769 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:04.765+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:04.779 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:04.774+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:04.927 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:04.923+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:05.088 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:05.083+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:05.260 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:05.256+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:05.661 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:05.657+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:05.789 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:05.774+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:05.821 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:05.815+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:05.961 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:05.956+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:06.056 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:06.051+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:06.258 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:06.253+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:06.639 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:06.634+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:06.763 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:06.759+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:06.798 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:06.792+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:06.939 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:06.935+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:07.060 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:07.056+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:07.267 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:07.263+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:07.676 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:07.672+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:07.792 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:07.787+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:07.841 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:07.836+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:07.952 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:07.946+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:08.089 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:08.084+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:08.241 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:08.237+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:08.669 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:08.665+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:08.766 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:08.761+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:08.867 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:08.862+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:08.953 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:08.949+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:09.090 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:09.085+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:09.238 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:09.233+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:09.711 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:09.706+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:09.727 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:09.722+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:09.831 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:09.827+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:09.912 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:09.907+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:10.066 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:10.061+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:10.242 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:10.238+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:10.756 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:10.751+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:10.764 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:10.759+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:10.801 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:10.796+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:10.902 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:10.897+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:11.088 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:11.084+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:11.289 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:11.285+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:11.767 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:11.762+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:11.780 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:11.775+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:11.823 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:11.819+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:11.928 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:11.923+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:12.105 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:12.099+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:12.251 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:12.246+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:12.734 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:12.729+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:12.760 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:12.756+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:12.868 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:12.863+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:12.977 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:12.972+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:13.149 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:13.143+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:13.244 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:13.239+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:13.718 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:13.714+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:13.789 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:13.785+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:13.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:13.861+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:14.027 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:14.023+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:14.185 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:14.180+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:14.269 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:14.264+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:14.717 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:14.713+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:14.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:14.763+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:14.893 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:14.887+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:15.060 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:15.055+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:15.177 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:15.173+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:15.232 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:15.227+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:15.771 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:15.767+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:15.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:15.771+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:15.868 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:15.864+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:16.044 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:16.039+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:16.178 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:16.173+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:16.246 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:16.242+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:16.722 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:31:16.718+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e543 v543) 2019-12-09T20:31:16.748 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:16.744+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:16.806 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:16.802+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:16.849 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:16.846+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:16.999 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:16.993+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:17.197 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:17.193+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:17.209 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:17.204+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:17.759 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:17.755+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:17.768 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:17.759+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:17.894 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:17.890+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:18.049 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:18.045+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:18.157 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:18.148+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:18.173 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:18.169+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:18.755 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:18.750+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:18.773 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:18.768+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:18.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:18.878+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:19.032 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:19.027+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:19.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:19.136+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:19.194 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:19.191+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:19.714 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:19.709+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:19.775 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:19.770+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:19.848 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:19.844+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:20.017 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:20.012+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:20.111 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:20.106+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:20.168 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:20.163+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:20.763 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:20.759+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:20.766 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:20.761+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:20.828 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:20.823+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:20.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:20.993+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:21.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:21.123+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:21.176 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:21.171+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:21.738 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:21.732+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:21.744 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:31:21.741+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e543 v543) 2019-12-09T20:31:21.799 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:21.794+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:21.854 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:21.850+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:22.012 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:22.007+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:22.141 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:22.137+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:22.208 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:22.203+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:22.711 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:22.704+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:22.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:22.763+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:22.819 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:22.815+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:22.975 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:22.972+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:23.126 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:23.122+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:23.230 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:23.225+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:23.692 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:23.687+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:23.797 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:23.792+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:23.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:23.859+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:23.982 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:23.978+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:24.128 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:24.124+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:24.229 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:24.225+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:24.733 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:24.728+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:24.762 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:24.757+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:24.872 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:24.868+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:24.986 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:24.982+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:25.122 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:25.117+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:25.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:25.250+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:25.734 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:25.729+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:25.775 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:25.771+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:25.924 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:25.920+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:25.992 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:25.988+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:31:26.153 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:31:26.150+0000 1694f700 -1 osd.1 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:58.404975+0000 front 2019-12-09T20:29:59.188071+0000 (oldest deadline 2019-12-09T20:30:42.086425+0000) 2019-12-09T20:31:26.254 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:31:26.250+0000 1694f700 -1 osd.7 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834358+0000 front 2019-12-09T20:30:01.727359+0000 (oldest deadline 2019-12-09T20:30:45.750447+0000) 2019-12-09T20:31:26.778 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:31:26.774+0000 1694f700 -1 osd.6 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.834639+0000 front 2019-12-09T20:30:01.424679+0000 (oldest deadline 2019-12-09T20:30:43.071269+0000) 2019-12-09T20:31:26.802 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:31:26.798+0000 1694f700 -1 osd.5 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:29:59.583999+0000 front 2019-12-09T20:30:00.347435+0000 (oldest deadline 2019-12-09T20:30:43.610252+0000) 2019-12-09T20:31:26.832 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:31:26.826+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 44sec e543 v543) 2019-12-09T20:31:26.952 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:31:26.949+0000 1694f700 -1 osd.2 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:01.127004+0000 front 2019-12-09T20:30:01.017636+0000 (oldest deadline 2019-12-09T20:30:45.634394+0000) 2019-12-09T20:31:26.997 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:31:26.994+0000 1694f700 -1 osd.0 543 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:30:00.562255+0000 front 2019-12-09T20:30:00.885640+0000 (oldest deadline 2019-12-09T20:30:44.551424+0000) 2019-12-09T20:32:12.089 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:32:11.943+0000 31f87700 -1 osd.5 547 build_incremental_map_msg missing incremental map 547 2019-12-09T20:32:12.253 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:32:12.194+0000 33f8b700 -1 osd.5 547 build_incremental_map_msg missing incremental map 547 2019-12-09T20:32:12.448 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:32:12.397+0000 3478c700 -1 osd.5 547 build_incremental_map_msg missing incremental map 547 2019-12-09T20:33:16.023 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:16.018+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:17.068 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:17.064+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:17.708 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:17.704+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:17.906 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:17.902+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:18.047 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:18.043+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:18.255 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:18.252+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:18.714 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:18.711+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:18.958 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:18.954+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:19.064 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:19.059+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:19.219 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:19.215+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:19.695 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:19.692+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:19.918 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:19.915+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:20.027 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:20.022+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:20.185 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:20.181+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:20.358 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:20.352+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:20.664 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:20.661+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:20.909 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:20.905+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:21.015 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:21.010+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:21.218 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:21.215+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:21.231 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:21.226+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:21.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:21.400+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:21.635 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:21.632+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:21.944 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:21.941+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:22.035 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:22.031+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:22.213 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:22.209+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:22.284 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:22.279+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:22.387 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:22.384+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:22.615 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:22.609+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:22.932 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:22.929+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:23.052 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:23.047+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:23.237 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:23.233+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:23.313 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:23.308+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:23.389 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:23.384+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:23.587 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:23.583+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:23.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:23.932+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:24.030 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:24.027+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:24.285 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:24.282+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:24.293 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:24.289+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:24.427 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:24.423+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:24.545 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:24.541+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:24.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:24.904+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:25.019 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:25.015+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:25.251 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:25.246+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:25.307 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:25.304+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:25.457 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:25.452+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:25.553 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:25.549+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:25.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:25.942+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:26.053 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:26.049+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:26.257 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:26.253+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:26.312 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:26.308+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:26.462 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:26.458+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:26.539 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:26.534+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:26.905 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:26.902+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:27.106 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:27.102+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:27.224 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:27.218+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:27.348 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:27.345+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:27.438 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:27.434+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:27.582 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:27.577+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:27.861 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:27.857+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:28.122 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:28.118+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:28.202 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:28.197+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:28.350 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:28.345+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:28.469 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:28.465+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:28.636 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:28.630+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:28.876 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:28.871+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:29.106 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:29.101+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:29.252 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:29.247+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:29.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:29.342+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:29.492 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:29.488+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:29.634 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:29.629+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:29.909 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:29.905+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:30.090 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:30.086+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:30.293 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:30.287+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:30.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:30.333+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:30.479 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:30.472+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:30.649 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:30.644+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:30.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:30.865+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:31.081 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:31.076+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:31.341 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:31.337+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:31.347 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:31.342+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:31.456 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:31.451+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:31.631 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:31.624+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:31.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:31.898+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:32.110 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:32.106+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:32.323 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:32.318+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:32.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:32.360+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:32.441 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:32.438+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:32.598 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:32.594+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:32.891 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:32.886+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:33.133 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:33.129+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:33.356 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:33.350+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:33.359 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:33.354+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:33.439 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:33.435+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:33.630 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:33.626+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:33.917 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:33.913+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:34.111 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:34.107+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:34.371 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:34.367+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:34.407 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:34.402+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:34.490 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:34.485+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:34.653 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:34.649+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:34.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:34.920+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:35.158 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:35.154+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:35.336 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:35.330+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:35.374 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:35.370+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:35.501 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:35.497+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:35.651 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:35.647+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:35.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:35.911+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:36.139 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:36.135+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:36.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:36.343+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:36.374 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:36.369+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:36.538 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:36.534+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:36.652 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:36.648+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:36.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:36.872+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:37.171 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:37.167+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:37.302 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:37.298+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:37.330 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:37.326+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:37.515 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:37.510+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:37.658 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:37.654+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:37.895 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:37.891+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:38.128 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:38.124+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:38.263 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:38.258+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:38.316 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:38.312+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:38.491 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:38.487+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:38.699 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:38.695+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:38.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:38.845+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:39.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:39.149+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:39.272 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:39.267+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:39.327 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:39.323+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:39.529 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:39.526+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:39.659 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:39.654+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:39.808 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:39.799+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:40.119 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:40.114+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:40.245 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:40.241+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:40.352 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:40.348+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:40.494 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:40.490+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:40.675 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:40.671+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:40.797 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:40.791+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:41.117 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:41.113+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:41.240 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:41.236+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:41.310 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:41.305+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:41.533 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:41.529+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:41.719 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:41.714+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:41.797 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:41.792+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:42.159 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:42.154+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:42.234 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:42.230+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:42.335 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:42.331+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:42.523 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:42.519+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:42.673 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:42.669+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:42.781 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:42.775+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:43.116 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:43.111+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:43.265 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:43.261+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:43.342 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:43.337+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:43.524 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:43.520+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:43.670 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:43.665+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:43.816 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:43.810+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:44.120 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:44.116+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:44.312 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:44.308+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:44.316 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:44.312+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:44.548 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:44.544+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:44.676 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:44.673+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:44.849 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:44.844+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:45.166 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:45.161+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:45.286 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:45.281+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:45.311 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:45.306+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:45.564 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:45.559+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:45.655 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:45.650+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:45.864 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:45.860+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:46.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:46.194+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:46.275 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:46.271+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:46.329 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:46.324+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:46.546 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:46.542+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:46.666 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:46.662+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:46.876 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:46.869+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:47.214 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:47.209+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:47.319 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:47.315+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:47.379 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:47.375+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:47.562 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:47.557+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:47.638 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:47.634+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:47.844 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:47.839+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:48.239 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:48.235+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:48.303 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:48.298+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:48.381 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:48.377+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:48.608 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:48.605+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:48.689 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:48.685+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:48.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:48.882+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:49.197 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:49.193+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:49.304 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:49.300+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:49.414 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:49.411+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:49.582 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:49.579+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:49.684 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:49.681+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:49.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:49.896+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:50.162 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:50.158+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:50.286 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:50.280+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:50.379 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:50.375+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:50.543 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:50.538+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:50.705 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:50.693+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:50.925 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:50.920+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:51.167 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:51.163+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:51.336 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:51.330+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:51.422 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:51.418+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:51.585 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:51.580+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:51.714 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:51.710+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:51.977 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:51.972+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:52.160 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:52.155+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:52.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:52.292+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:52.410 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:33:52.407+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e550 v550) 2019-12-09T20:33:52.432 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:52.427+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:52.592 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:52.588+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:52.750 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:52.746+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:52.945 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:52.941+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:53.139 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:53.135+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:53.321 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:53.315+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:53.400 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:53.395+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:53.612 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:53.608+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:53.751 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:53.747+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:53.962 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:53.960+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:54.146 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:54.141+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:54.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:54.311+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:54.414 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:54.411+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:54.569 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:54.564+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:54.802 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:54.797+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:54.991 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:54.988+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:55.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:55.174+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:55.361 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:55.357+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:55.375 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:55.371+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:55.609 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:55.604+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:55.790 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:55.786+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:55.986 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:55.983+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:56.159 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:56.155+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:56.340 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:56.336+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:56.428 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:56.424+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:56.657 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:33:56.652+0000 1694f700 -1 osd.6 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.161272+0000 front 2019-12-09T20:32:34.158925+0000 (oldest deadline 2019-12-09T20:33:20.004306+0000) 2019-12-09T20:33:56.841 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:33:56.838+0000 1694f700 -1 osd.1 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:32.279636+0000 front 2019-12-09T20:32:32.326812+0000 (oldest deadline 2019-12-09T20:33:16.889112+0000) 2019-12-09T20:33:56.998 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:33:56.992+0000 1694f700 -1 osd.2 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.696841+0000 front 2019-12-09T20:32:33.739404+0000 (oldest deadline 2019-12-09T20:33:17.140858+0000) 2019-12-09T20:33:57.215 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:33:57.210+0000 1694f700 -1 osd.5 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:35.889025+0000 front 2019-12-09T20:32:31.822377+0000 (oldest deadline 2019-12-09T20:33:15.847947+0000) 2019-12-09T20:33:57.395 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:33:57.386+0000 1694f700 -1 osd.0 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:33.169129+0000 front 2019-12-09T20:32:33.167354+0000 (oldest deadline 2019-12-09T20:33:17.585249+0000) 2019-12-09T20:33:57.427 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:33:57.423+0000 1694f700 -1 osd.7 550 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:32:34.828703+0000 front 2019-12-09T20:32:34.899691+0000 (oldest deadline 2019-12-09T20:33:20.702665+0000) 2019-12-09T20:33:57.497 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:33:57.490+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 46sec e550 v550) 2019-12-09T20:34:32.784 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:24.635 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:32.837 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:24.688 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:32.842 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:24.667 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:32.844 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:24.669 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:32.848 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:24.673 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:32.868 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:24.693 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:33.078 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:24.929 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:33.109 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:24.934 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:33.130 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:24.955 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:42.799 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:34:42.782+0000 11caf700 -1 osd.1 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:42.815 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:34:42.771+0000 114ae700 -1 osd.0 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:42.912 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:34:42.842+0000 3578e700 -1 osd.0 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:42.975 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:34:42.795+0000 32b88700 -1 osd.1 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.061 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:34:42.876+0000 33389700 -1 osd.1 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.094 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:34:43.061+0000 36f91700 -1 osd.0 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.298 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:34:43.150+0000 3438b700 -1 osd.1 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.299 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:34:43.200+0000 33f8b700 -1 osd.0 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.301 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:34:43.179+0000 37f93700 -1 osd.5 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.336 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:34:43.224+0000 3478c700 -1 osd.5 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:43.414 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:34:43.405+0000 38794700 -1 osd.0 554 build_incremental_map_msg missing incremental map 554 2019-12-09T20:34:46.257 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:38.108 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.267 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:38.117 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.276 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:38.101 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.279 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:38.104 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.288 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:38.113 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.295 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:38.120 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.315 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:13:38.166 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.326 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:13:38.152 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:34:46.372 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:13:38.197 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:35:46.817 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:46.812+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:47.462 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:47.455+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:47.574 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:47.570+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:47.798 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:47.793+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:48.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:48.236+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:48.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:48.441+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:48.655 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:48.571+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:48.799 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:48.796+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:49.276 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:49.271+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:49.300 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:49.285+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:49.403 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:49.397+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:49.556 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:49.553+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:49.784 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:49.779+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:50.271 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:50.267+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:50.286 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:50.283+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:50.293 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:50.289+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:50.369 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:50.360+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:50.536 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:50.532+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:50.748 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:50.744+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:51.296 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:51.291+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:51.307 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:51.303+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:51.314 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:51.310+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:51.374 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:51.369+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:51.494 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:51.490+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:51.762 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:51.755+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:52.285 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:52.279+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:52.304 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:52.299+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:52.321 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:52.318+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:52.376 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:52.371+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:52.476 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:52.472+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:52.767 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:52.763+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:53.262 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:53.257+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:53.288 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:53.286+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:53.310 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:53.306+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:53.384 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:53.379+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:53.439 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:53.435+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:53.722 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:53.718+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:54.259 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:54.254+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:54.278 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:54.274+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:54.346 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:54.342+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:54.350 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:54.346+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:54.461 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:54.456+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:54.742 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:54.737+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:55.240 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:55.236+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:55.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:55.292+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:55.303 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:55.299+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:55.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:55.309+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:55.486 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:55.482+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:55.793 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:55.790+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:56.221 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:56.217+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:56.264 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:56.261+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:56.264 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:56.257+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:56.350 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:56.347+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:56.470 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:56.466+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:56.825 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:56.822+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:57.230 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:57.226+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:57.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:57.266+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:57.297 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:57.293+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:57.310 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:57.306+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:57.499 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:57.496+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:57.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:57.861+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:58.224 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:58.219+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:58.239 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:58.233+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:58.279 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:58.273+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:58.286 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:58.280+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:58.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:58.518+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:58.848 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:58.844+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:35:59.192 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:35:59.188+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:35:59.221 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:35:59.216+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:35:59.236 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:35:59.231+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:35:59.338 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:35:59.333+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:35:59.488 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:35:59.483+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:35:59.865 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:35:59.861+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:00.201 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:00.191+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:00.201 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:00.196+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:00.226 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:00.220+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:00.353 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:00.348+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:00.533 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:00.529+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:00.878 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:00.873+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:01.181 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:01.176+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:01.238 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:01.233+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:01.247 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:01.242+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:01.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:01.360+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:01.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:01.525+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:01.911 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:01.908+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:02.221 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:02.217+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:02.251 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:02.246+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:02.271 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:02.267+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:02.406 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:02.397+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:02.555 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:02.550+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:02.930 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:02.925+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:03.239 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:03.234+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:03.254 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:03.250+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:03.264 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:03.260+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:03.421 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:03.418+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:03.546 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:03.542+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:03.900 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:03.896+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:04.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:04.265+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:04.306 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:04.302+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:04.318 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:04.314+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:04.420 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:04.417+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:04.501 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:04.497+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:04.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:04.868+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:05.241 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:05.237+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:05.337 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:05.331+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:05.346 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:05.342+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:05.424 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:05.420+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:05.540 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:05.533+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:05.830 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:05.825+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:06.295 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:06.290+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:06.344 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:06.333+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:06.345 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:06.337+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:06.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:06.408+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:06.529 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:06.524+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:06.884 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:06.879+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:07.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:07.315+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:07.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:07.319+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:07.376 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:07.372+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:07.383 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:07.379+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:07.492 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:07.488+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:07.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:07.901+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:08.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:08.328+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:08.337 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:08.334+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:08.385 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:08.380+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:08.400 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:08.395+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:08.552 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:08.540+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:08.947 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:08.944+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:09.302 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:09.298+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:09.358 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:09.355+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:09.395 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:09.391+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:09.414 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:09.409+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:09.553 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:09.545+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:09.967 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:09.962+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:10.265 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:10.260+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:10.331 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:10.328+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:10.371 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:10.367+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:10.432 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:10.429+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:10.593 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:10.587+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:10.964 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:10.959+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:11.289 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:11.285+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:11.314 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:11.309+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:11.384 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:11.378+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:11.393 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:11.389+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:11.584 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:11.579+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:12.014 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:12.004+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:12.335 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:12.331+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:12.337 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:12.334+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:12.351 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:12.347+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:12.396 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:12.392+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:12.552 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:12.547+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:13.034 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:13.029+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:13.364 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:13.360+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:13.370 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:13.365+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:13.375 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:13.370+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:13.378 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:13.374+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:13.561 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:13.556+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:13.989 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:13.986+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:14.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:14.360+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:14.384 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:14.380+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:14.394 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:14.390+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:14.399 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:14.396+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:14.592 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:14.587+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:14.991 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:14.987+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:15.378 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:15.374+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:15.381 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:15.377+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:15.411 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:15.407+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:15.445 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:15.440+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:15.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:15.561+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:16.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:16.005+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:16.395 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:16.386+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:16.423 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:16.418+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:16.431 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:16.427+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:16.466 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:16.461+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:16.566 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:16.561+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:16.972 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:16.967+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:17.403 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:17.398+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:17.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:17.417+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:17.455 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:17.450+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:17.462 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:17.459+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:17.554 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:17.550+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:18.028 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:18.020+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:18.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:18.409+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:18.433 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:18.429+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:18.438 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:18.434+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:18.503 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:18.499+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:18.579 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:18.575+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:19.057 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:19.051+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:19.428 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:19.424+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:19.440 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:19.436+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:19.470 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:19.465+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:19.525 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:19.522+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:19.534 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:19.529+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:20.095 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:20.091+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:20.414 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:20.410+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:20.470 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:20.465+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:20.491 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:20.482+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:20.505 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:20.501+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:20.555 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:20.552+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:21.085 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:21.080+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:21.466 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:21.462+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:21.472 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:21.466+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:21.493 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:21.483+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:21.501 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:21.497+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:21.568 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:21.565+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:22.110 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:22.106+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:22.457 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:22.448+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:22.470 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:22.466+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:22.482 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:22.478+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:22.500 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:22.495+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:22.601 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:22.591+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:23.136 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:23.130+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:23.260 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:36:23.254+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e557 v557) 2019-12-09T20:36:23.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:23.417+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:23.429 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:23.425+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:23.448 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:23.445+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:23.479 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:23.476+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:23.575 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:23.572+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:24.179 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:24.174+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:24.393 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:24.391+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:24.406 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:24.402+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:24.422 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:24.418+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:24.501 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:24.498+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:24.621 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:24.616+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:25.159 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:25.154+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:25.366 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:25.361+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:25.447 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:25.443+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:25.460 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:25.457+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:25.518 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:25.515+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:25.618 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:25.614+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:26.181 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:26.177+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:26.388 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:26.385+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:26.419 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:26.415+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:26.427 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:26.424+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:26.554 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:26.548+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:26.628 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:26.621+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:27.193 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:27.188+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:27.362 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:27.358+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:27.413 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:27.407+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:36:27.428 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:27.425+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:27.531 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:36:27.527+0000 1694f700 -1 osd.0 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.485027+0000 front 2019-12-09T20:35:05.189179+0000 (oldest deadline 2019-12-09T20:35:49.544377+0000) 2019-12-09T20:36:27.667 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:36:27.663+0000 1694f700 -1 osd.5 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:05.573994+0000 front 2019-12-09T20:35:05.572632+0000 (oldest deadline 2019-12-09T20:35:47.729037+0000) 2019-12-09T20:36:28.214 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:36:28.210+0000 1694f700 -1 osd.6 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.146924+0000 front 2019-12-09T20:35:03.539108+0000 (oldest deadline 2019-12-09T20:35:46.622560+0000) 2019-12-09T20:36:28.359 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:36:28.354+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e557 v557) 2019-12-09T20:36:28.419 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:36:28.415+0000 1694f700 -1 osd.2 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:03.963453+0000 front 2019-12-09T20:35:03.956564+0000 (oldest deadline 2019-12-09T20:35:48.523905+0000) 2019-12-09T20:36:28.435 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:36:28.411+0000 1694f700 -1 osd.1 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:01.705723+0000 front 2019-12-09T20:35:01.860436+0000 (oldest deadline 2019-12-09T20:35:46.987758+0000) 2019-12-09T20:36:28.440 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:36:28.437+0000 1694f700 -1 osd.7 557 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:35:04.443341+0000 front 2019-12-09T20:35:04.468865+0000 (oldest deadline 2019-12-09T20:35:47.149089+0000) 2019-12-09T20:37:12.348 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:37:12.200+0000 32b88700 -1 osd.1 561 build_incremental_map_msg missing incremental map 561 2019-12-09T20:37:12.557 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:37:12.378+0000 33389700 -1 osd.1 561 build_incremental_map_msg missing incremental map 561 2019-12-09T20:37:12.648 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:37:12.456+0000 38393700 -1 osd.1 561 build_incremental_map_msg missing incremental map 561 2019-12-09T20:37:15.187 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:37:15.078+0000 33f8b700 -1 osd.3 563 build_incremental_map_msg missing incremental map 563 2019-12-09T20:37:15.732 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:37:15.414+0000 31786700 -1 osd.3 563 build_incremental_map_msg missing incremental map 563 2019-12-09T20:37:16.924 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:37:16.915+0000 37f93700 -1 osd.3 563 build_incremental_map_msg missing incremental map 563 2019-12-09T20:37:17.920 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:37:17.672+0000 32f89700 -1 osd.3 564 build_incremental_map_msg missing incremental map 563 2019-12-09T20:38:17.582 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:17.578+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:17.596 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:17.590+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:17.629 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:17.625+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:18.614 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:18.608+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:18.629 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:18.623+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:18.634 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:18.630+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:19.509 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:19.505+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:19.596 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:19.593+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:19.650 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:19.646+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:19.673 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:19.656+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:20.508 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:20.503+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:20.623 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:20.619+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:20.654 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:20.650+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:20.664 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:20.659+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:21.527 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:21.524+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:21.625 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:21.621+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:21.628 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:21.624+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:21.651 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:21.647+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:21.938 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:21.934+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:22.251 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:22.248+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:22.523 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:22.519+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:22.599 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:22.595+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:22.612 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:22.608+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:22.621 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:22.617+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:22.925 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:22.921+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:23.221 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:23.217+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:23.515 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:23.510+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:23.581 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:23.576+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:23.612 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:23.608+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:23.624 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:23.616+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:23.913 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:23.909+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:24.207 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:24.204+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:24.511 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:24.508+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:24.573 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:24.568+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:24.605 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:24.601+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:24.644 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:24.640+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:24.896 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:24.893+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:25.160 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:25.157+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:25.526 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:25.522+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:25.564 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:25.559+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:25.570 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:25.565+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:25.645 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:25.634+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:25.942 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:25.937+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:26.158 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:26.151+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:26.542 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:26.538+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:26.559 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:26.555+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:26.579 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:26.576+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:26.642 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:26.639+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:26.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:26.902+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:27.169 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:27.160+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:27.510 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:27.506+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:27.589 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:27.586+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:27.630 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:27.627+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:27.633 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:27.630+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:27.873 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:27.870+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:28.143 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:28.140+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:28.523 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:28.519+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:28.561 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:28.556+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:28.623 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:28.618+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:28.673 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:28.669+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:28.864 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:28.859+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:29.130 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:29.126+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:29.531 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:29.520+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:29.602 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:29.597+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:29.625 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:29.620+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:29.706 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:29.703+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:29.878 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:29.873+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:30.164 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:30.160+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:30.578 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:30.575+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:30.618 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:30.614+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:30.650 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:30.646+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:30.732 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:30.727+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:30.881 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:30.877+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:31.152 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:31.148+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:31.584 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:31.580+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:31.629 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:31.624+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:31.688 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:31.682+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:31.755 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:31.751+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:31.838 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:31.834+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:32.201 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:32.196+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:32.556 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:32.551+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:32.647 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:32.643+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:32.670 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:32.665+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:32.750 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:32.745+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:32.892 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:32.883+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:33.234 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:33.230+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:33.599 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:33.596+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:33.655 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:33.651+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:33.660 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:33.655+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:33.764 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:33.761+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:33.866 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:33.862+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:34.238 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:34.233+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:34.645 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:34.640+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:34.650 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:34.646+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:34.665 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:34.661+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:34.793 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:34.780+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:34.867 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:34.863+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:35.249 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:35.244+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:35.610 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:35.606+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:35.686 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:35.682+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:35.706 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:35.702+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:35.807 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:35.803+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:35.915 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:35.911+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:36.249 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:36.244+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:36.593 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:36.589+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:36.663 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:36.658+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:36.688 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:36.684+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:36.792 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:36.787+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:36.893 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:36.888+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:37.254 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:37.249+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:37.619 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:37.614+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:37.659 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:37.655+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:37.684 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:37.680+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:37.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:37.796+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:37.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:37.868+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:38.262 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:38.258+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:38.648 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:38.643+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:38.651 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:38.646+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:38.727 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:38.722+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:38.804 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:38.800+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:38.830 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:38.826+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:39.284 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:39.279+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:39.633 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:39.628+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:39.633 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:39.628+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:39.744 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:39.740+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:39.865 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:39.859+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:39.869 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:39.865+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:40.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:40.303+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:40.623 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:40.618+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:40.677 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:40.673+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:40.732 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:40.728+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:40.855 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:40.851+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:40.888 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:40.883+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:41.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:41.297+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:41.639 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:41.634+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:41.710 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:41.706+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:41.788 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:41.783+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:41.883 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:41.879+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:41.884 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:41.878+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:42.335 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:42.331+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:42.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:42.636+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:42.707 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:42.704+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:42.835 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:42.830+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:42.897 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:42.892+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:42.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:42.902+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:43.338 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:43.333+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:43.693 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:43.689+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:43.751 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:43.747+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:43.830 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:43.825+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:43.883 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:43.878+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:43.956 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:43.952+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:44.369 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:44.364+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:44.675 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:44.670+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:44.726 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:44.722+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:44.837 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:44.834+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:44.851 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:44.845+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:44.960 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:44.956+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:45.376 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:45.370+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:45.694 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:45.688+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:45.774 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:45.771+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:45.839 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:45.836+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:45.848 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:45.843+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:45.941 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:45.938+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:46.401 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:46.396+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:46.724 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:46.720+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:46.793 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:46.789+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:46.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:46.879+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:46.892 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:46.887+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:46.937 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:46.931+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:47.409 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:47.404+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:47.717 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:47.712+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:47.783 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:47.779+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:47.850 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:47.846+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:47.905 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:47.900+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:47.932 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:47.927+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:48.363 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:48.359+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:48.733 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:48.728+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:48.744 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:48.740+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:48.824 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:48.818+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:48.864 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:48.859+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:48.967 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:48.963+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:49.042 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:38:49.038+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e565 v565) 2019-12-09T20:38:49.361 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:49.356+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:49.722 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:49.718+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:49.768 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:49.764+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:49.800 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:49.797+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:49.906 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:49.901+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:49.934 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:49.930+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:50.385 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:50.381+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:50.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:50.725+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:50.751 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:50.744+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:50.844 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:50.839+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:50.924 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:50.918+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:50.933 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:50.929+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:51.409 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:51.403+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:51.746 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:51.742+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:51.761 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:51.757+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:51.813 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:51.807+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:51.937 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:51.932+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:51.953 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:51.947+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:52.398 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:52.393+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:52.761 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:52.756+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:52.785 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:52.780+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:52.792 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:52.787+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:52.930 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:52.927+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:52.949 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:52.944+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:53.376 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:53.372+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:53.715 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:53.711+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:53.810 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:53.806+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:53.830 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:53.826+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:53.903 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:53.899+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:53.996 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:53.992+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:54.057 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:38:54.054+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e565 v565) 2019-12-09T20:38:54.359 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:54.356+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:54.674 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:54.669+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:54.796 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:54.792+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:54.851 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:54.847+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:54.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:54.943+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:54.982 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:54.979+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:55.322 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:55.317+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:55.649 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:55.646+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:55.811 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:55.808+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:55.863 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:55.859+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:55.941 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:55.938+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:56.024 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:56.020+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:56.300 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:56.297+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:56.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:56.685+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:56.795 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:56.791+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:56.822 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:56.818+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:56.987 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:56.985+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:57.078 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:57.074+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:57.262 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:57.258+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:57.699 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:57.694+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:57.802 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:57.799+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:57.861 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:57.851+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:58.027 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:58.024+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:58.120 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:58.116+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:58.269 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:58.265+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:38:58.719 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:38:58.715+0000 1694f700 -1 osd.5 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:31.989453+0000 front 2019-12-09T20:37:31.990965+0000 (oldest deadline 2019-12-09T20:38:17.225923+0000) 2019-12-09T20:38:58.781 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:38:58.773+0000 1694f700 -1 osd.0 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.092793+0000 front 2019-12-09T20:37:36.012594+0000 (oldest deadline 2019-12-09T20:38:19.312544+0000) 2019-12-09T20:38:58.856 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:38:58.852+0000 1694f700 -1 osd.7 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:32.419809+0000 front 2019-12-09T20:37:32.500757+0000 (oldest deadline 2019-12-09T20:38:17.084863+0000) 2019-12-09T20:38:59.081 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:38:59.076+0000 1694f700 -1 osd.2 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.939215+0000 front 2019-12-09T20:37:32.849935+0000 (oldest deadline 2019-12-09T20:38:16.881955+0000) 2019-12-09T20:38:59.138 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:38:59.134+0000 1694f700 -1 osd.6 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:36.093662+0000 front 2019-12-09T20:37:36.051358+0000 (oldest deadline 2019-12-09T20:38:21.822204+0000) 2019-12-09T20:38:59.157 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:38:59.147+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e565 v565) 2019-12-09T20:38:59.277 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:38:59.272+0000 1694f700 -1 osd.1 565 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:37:35.977586+0000 front 2019-12-09T20:37:35.747305+0000 (oldest deadline 2019-12-09T20:38:21.352384+0000) 2019-12-09T20:39:00.649 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:17:52.500 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.664 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:17:52.515 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.685 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:17:52.510 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.689 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:17:52.514 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.691 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:17:52.514 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.696 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:17:52.519 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:00.937 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:17:52.788 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:01.169 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:17:52.991 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:01.209 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:17:53.034 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:39:46.266 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:39:46.191+0000 35f8f700 -1 osd.5 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.454 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:39:46.296+0000 3578e700 -1 osd.0 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.462 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:39:46.284+0000 37f93700 -1 osd.5 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.669 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:39:46.609+0000 3478c700 -1 osd.5 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.691 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:39:46.486+0000 3378a700 -1 osd.0 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.788 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:39:46.694+0000 33f8b700 -1 osd.0 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.796 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:39:46.541+0000 31f87700 -1 osd.2 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.844 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:39:46.819+0000 3478c700 -1 osd.0 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.891 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:39:46.703+0000 34b8c700 -1 osd.2 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:39:46.895 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:39:46.723+0000 33f8b700 -1 osd.2 569 build_incremental_map_msg missing incremental map 569 2019-12-09T20:40:53.092 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:53.088+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:53.984 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:53.977+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:54.110 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:54.106+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:54.165 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:54.159+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:54.381 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:54.378+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:54.947 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:54.944+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:55.077 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:55.072+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:55.179 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:55.174+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:55.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:55.375+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:55.946 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:55.942+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:56.086 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:56.082+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:56.172 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:56.167+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:56.406 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:56.401+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:56.686 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:40:56.683+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:40:56.974 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:56.967+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:57.060 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:57.056+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:57.169 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:57.164+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:57.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:57.375+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:57.725 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:40:57.720+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:40:57.936 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:57.932+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:58.025 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:58.021+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:58.136 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:58.131+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:58.409 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:58.405+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:58.717 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:40:58.713+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:40:58.906 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:58.901+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:40:59.063 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:40:59.059+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:40:59.161 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:40:59.154+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:40:59.437 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:40:59.431+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:40:59.767 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:40:59.763+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:40:59.917 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:40:59.913+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:00.037 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:00.033+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:00.199 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:00.194+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:00.457 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:41:00.449+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:41:00.786 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:00.781+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:00.906 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:00.900+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:01.069 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:01.059+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:01.188 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:01.183+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:01.463 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:41:01.458+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:41:01.808 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:01.803+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:01.871 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:01.866+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:02.022 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:02.018+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:02.213 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:02.209+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:02.423 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:41:02.419+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:41:02.824 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:02.820+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:02.825 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:02.821+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:03.025 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:03.021+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:03.191 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:03.187+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:03.436 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:41:03.431+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:41:03.808 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:03.803+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:03.879 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:03.874+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:04.038 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:04.033+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:04.210 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:04.205+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:04.487 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:41:04.483+0000 1694f700 -1 osd.1 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.644129+0000 front 2019-12-09T20:40:08.704337+0000 (oldest deadline 2019-12-09T20:40:53.854588+0000) 2019-12-09T20:41:04.763 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:04.759+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:04.908 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:04.903+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:05.005 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:05.001+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:05.166 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:05.162+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:05.720 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:05.715+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:05.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:05.866+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:05.977 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:05.972+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:06.139 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:06.135+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:06.738 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:06.734+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:06.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:06.867+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:06.936 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:06.932+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:07.138 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:07.133+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:07.768 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:07.764+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:07.846 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:07.842+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:07.981 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:07.977+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:08.174 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:08.170+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:08.768 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:08.764+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:08.808 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:08.803+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:08.995 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:08.990+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:09.188 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:09.183+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:09.766 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:09.763+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:09.837 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:09.833+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:09.990 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:09.986+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:10.238 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:10.234+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:10.795 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:10.791+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:10.820 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:10.817+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:10.945 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:10.941+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:11.289 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:11.285+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:11.783 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:11.778+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:11.825 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:11.820+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:11.909 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:11.904+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:12.281 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:12.276+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:12.771 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:12.765+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:12.830 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:12.826+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:12.889 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:12.885+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:13.285 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:13.275+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:13.745 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:13.741+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:13.856 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:13.852+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:13.868 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:13.863+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:14.306 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:14.302+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:14.713 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:14.708+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:14.821 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:14.817+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:14.844 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:14.840+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:15.269 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:15.264+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:15.294 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:15.289+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:15.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:15.699+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:15.825 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:15.821+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:15.880 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:15.875+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:16.233 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:16.230+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:16.328 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:16.324+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:16.754 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:16.750+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:16.807 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:16.800+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:16.893 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:16.889+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:17.260 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:17.256+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:17.340 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:17.334+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:17.766 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:17.761+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:17.832 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:17.828+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:17.870 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:17.865+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:18.263 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:18.258+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:18.305 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:18.300+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:18.801 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:18.796+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:18.832 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:18.827+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:18.870 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:18.865+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:19.278 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:19.274+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:19.326 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:19.322+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:19.811 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:19.807+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:19.837 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:19.833+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:19.871 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:19.866+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:20.304 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:20.298+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:20.335 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:20.330+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:20.860 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:20.856+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:20.878 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:20.874+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:20.887 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:20.882+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:21.310 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:21.305+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:21.359 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:21.353+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:21.871 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:21.867+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:21.897 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:21.892+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:21.927 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:21.923+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:22.325 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:22.320+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:22.326 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:22.322+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:22.859 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:22.854+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:22.887 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:22.883+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:22.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:22.884+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:23.322 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:23.318+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:23.365 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:23.360+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:23.853 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:23.850+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:23.863 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:23.860+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:23.872 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:23.869+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:24.298 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:24.293+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:24.399 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:24.394+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:24.820 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:24.817+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:24.897 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:41:24.894+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e572 v572) 2019-12-09T20:41:24.904 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:24.901+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:24.907 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:24.902+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:25.355 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:25.347+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:25.385 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:25.380+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:25.813 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:25.810+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:25.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:25.898+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:25.941 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:25.937+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:26.330 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:26.327+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:26.435 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:26.431+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:26.799 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:26.795+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:26.918 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:26.914+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:26.940 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:26.937+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:27.364 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:27.356+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:27.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:27.441+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:27.835 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:27.831+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:27.909 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:27.905+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:27.944 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:27.940+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:28.389 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:28.384+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:28.449 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:28.445+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:28.866 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:28.862+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:28.916 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:28.912+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:28.922 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:28.919+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:29.386 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:29.382+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:29.480 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:29.477+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:29.863 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:29.859+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:29.882 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:29.878+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:29.918 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:41:29.915+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e572 v572) 2019-12-09T20:41:29.964 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:29.960+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:30.350 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:30.345+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:30.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:30.509+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:30.897 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:30.893+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:30.906 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:30.902+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:30.945 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:30.941+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:31.350 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:31.347+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:31.497 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:31.494+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:31.863 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:31.858+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:31.902 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:31.898+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:31.953 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:31.949+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:32.319 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:32.314+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:32.553 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:32.549+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:32.851 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:32.847+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:32.861 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:32.857+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:32.926 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:32.922+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:33.347 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:33.343+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:33.569 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:33.563+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:33.896 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:33.892+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:33.900 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:33.896+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:33.969 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:33.967+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:34.391 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:41:34.388+0000 1694f700 -1 osd.5 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:08.813020+0000 front 2019-12-09T20:40:08.821885+0000 (oldest deadline 2019-12-09T20:40:53.339427+0000) 2019-12-09T20:41:34.576 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:41:34.573+0000 1694f700 -1 osd.7 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.295212+0000 front 2019-12-09T20:40:10.095083+0000 (oldest deadline 2019-12-09T20:41:15.179751+0000) 2019-12-09T20:41:34.869 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:41:34.864+0000 1694f700 -1 osd.0 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.324503+0000 front 2019-12-09T20:40:10.334620+0000 (oldest deadline 2019-12-09T20:40:53.158400+0000) 2019-12-09T20:41:34.898 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:41:34.894+0000 1694f700 -1 osd.2 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.664525+0000 front 2019-12-09T20:40:10.692507+0000 (oldest deadline 2019-12-09T20:40:56.444142+0000) 2019-12-09T20:41:34.926 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:41:34.922+0000 1694f700 -1 osd.6 572 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:40:10.511445+0000 front 2019-12-09T20:40:10.663671+0000 (oldest deadline 2019-12-09T20:40:52.166030+0000) 2019-12-09T20:41:35.008 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:41:35.006+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 42sec e572 v572) 2019-12-09T20:42:19.686 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:19.631+0000 11caf700 -1 osd.3 577 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:19.844 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:19.833+0000 34f8d700 -1 osd.3 577 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:19.852 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:19.844+0000 34f8d700 -1 osd.3 577 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:19.967 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:19.927+0000 31786700 -1 osd.3 577 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:20.073 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.034+0000 37f93700 -1 osd.3 577 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:20.576 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.558+0000 31f87700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:20.612 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.594+0000 31f87700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:20.767 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.659+0000 32788700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:20.888 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.761+0000 31786700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:20.987 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:20.845+0000 31786700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:21.263 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:21.241+0000 33f8b700 -1 osd.3 578 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:21.289 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:21.242+0000 33f8b700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:21.584 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:21.573+0000 36f91700 -1 osd.3 578 build_incremental_map_msg missing incremental map 577 2019-12-09T20:42:21.585 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:42:21.577+0000 36f91700 -1 osd.3 578 build_incremental_map_msg missing incremental map 578 2019-12-09T20:42:55.586 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:47.437 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.591 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:47.442 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.629 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:47.454 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.632 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:47.457 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.634 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:47.460 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.637 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:47.462 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.939 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:47.765 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.956 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:47.781 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:42:55.990 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:47.841 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.800 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:52.650 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.809 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:52.660 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.814 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:52.640 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.815 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:52.641 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.823 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:52.649 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.824 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:52.649 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.862 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:21:52.687 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.867 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:21:52.693 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:00.879 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:21:52.730 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:43:23.629 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:23.625+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:23.766 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:23.763+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:23.769 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:23.765+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:24.433 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:24.429+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:24.618 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:24.613+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:24.771 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:24.768+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:24.804 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:24.799+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:25.459 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:25.453+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:25.613 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:25.611+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:25.784 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:25.781+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:25.788 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:25.785+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:26.451 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:26.447+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:26.590 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:26.587+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:26.594 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:26.590+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:26.750 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:26.747+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:26.787 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:26.784+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:27.407 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:27.402+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:27.565 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:27.561+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:27.583 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:27.579+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:27.730 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:27.727+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:27.766 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:27.762+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:28.459 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:28.454+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:28.564 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:28.561+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:28.591 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:28.587+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:28.716 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:28.712+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:28.763 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:28.759+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:29.491 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:29.485+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:29.597 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:29.593+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:29.614 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:29.610+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:29.770 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:29.766+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:29.792 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:29.788+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:30.456 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:30.451+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:30.616 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:30.611+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:30.663 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:30.660+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:30.806 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:30.803+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:30.819 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:30.816+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:31.478 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:31.475+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:31.628 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:31.625+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:31.645 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:31.641+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:31.763 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:31.759+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:31.855 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:31.852+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:32.439 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:32.434+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:32.620 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:32.616+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:32.640 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:32.637+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:32.781 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:32.777+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:32.887 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:32.883+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:33.445 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:33.441+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:33.593 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:33.588+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:33.679 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:33.674+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:33.799 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:33.795+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:33.926 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:33.922+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:34.452 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:34.447+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:34.592 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:34.587+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:34.680 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:34.672+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:34.824 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:34.820+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:34.923 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:34.918+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:35.449 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:35.445+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:35.608 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:35.604+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:35.660 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:35.656+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:35.853 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:35.849+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:35.971 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:35.968+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:36.443 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:36.439+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:36.607 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:36.603+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:36.667 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:36.663+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:36.883 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:36.879+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:36.982 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:36.977+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:37.430 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:37.426+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:37.646 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:37.642+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:37.652 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:37.648+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:37.928 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:37.923+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:38.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:38.021+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:38.418 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:38.413+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:38.624 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:38.619+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:38.682 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:38.677+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:38.899 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:38.895+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:39.014 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:39.007+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:39.437 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:39.433+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:39.652 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:39.647+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:39.686 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:39.682+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:39.950 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:39.945+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:40.017 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:40.006+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:40.391 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:40.387+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:40.644 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:40.640+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:40.733 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:40.730+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:40.925 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:40.922+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:41.039 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:41.035+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:41.409 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:41.406+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:41.621 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:41.617+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:41.725 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:41.721+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:41.977 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:41.974+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:42.025 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:42.018+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:42.463 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:42.460+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:42.639 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:42.635+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:42.706 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:42.702+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:43.008 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:43.005+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:43.026 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:43.021+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:43.431 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:43.427+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:43.685 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:43.680+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:43.740 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:43.735+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:44.000 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:43.996+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:44.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:43.996+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:44.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:44.467+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:44.672 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:44.667+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:44.766 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:44.761+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:44.997 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:44.993+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:45.006 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:45.000+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:45.447 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:45.443+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:45.702 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:45.691+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:45.719 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:45.716+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:46.006 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:46.002+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:46.027 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:46.024+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:46.453 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:46.448+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:46.745 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:46.740+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:46.761 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:46.757+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:47.042 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:47.036+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:47.053 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:47.049+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:47.498 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:47.494+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:47.758 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:47.753+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:47.786 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:47.782+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:48.026 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:48.022+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:48.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:48.047+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:48.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:48.510+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:48.542 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:48.537+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:48.772 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:48.765+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:48.802 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:48.797+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:48.991 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:48.987+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:49.068 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:49.064+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:49.508 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:49.504+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:49.529 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:49.522+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:49.745 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:49.741+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:49.770 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:49.766+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:50.019 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:50.015+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:50.077 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:50.070+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:50.524 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:50.521+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:50.557 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:50.552+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:50.753 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:50.749+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:50.823 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:50.810+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:51.027 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:51.022+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:51.033 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:51.027+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:51.541 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:51.537+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:51.601 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:51.596+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:51.774 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:51.770+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:51.810 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:51.806+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:52.005 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:52.000+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:52.047 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:52.042+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:52.514 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:52.510+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:52.635 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:52.630+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:52.774 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:52.770+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:52.860 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:52.856+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:53.027 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:53.023+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:53.039 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:53.035+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:53.471 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:53.467+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:53.626 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:53.621+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:53.749 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:53.745+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:53.889 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:53.883+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:53.998 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:53.993+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:54.023 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:54.020+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:54.521 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:54.517+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:54.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:54.636+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:54.722 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:54.718+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:54.894 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:54.889+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:55.006 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:55.002+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:55.059 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:55.054+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:55.553 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:55.548+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:55.675 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:55.671+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:55.709 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:55.697+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:55.817 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:43:55.814+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e580 v580) 2019-12-09T20:43:55.921 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:55.918+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:56.040 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:56.037+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:56.081 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:56.076+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:56.590 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:56.586+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:56.638 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:56.634+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:56.663 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:56.659+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:56.939 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:56.935+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:57.088 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:57.085+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:57.115 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:57.111+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:57.602 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:57.600+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:57.635 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:57.631+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:57.687 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:57.683+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:57.946 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:57.941+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:58.124 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:58.120+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:58.132 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:58.127+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:58.602 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:58.597+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:58.617 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:58.614+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:58.691 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:58.687+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:43:58.986 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:43:58.983+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:43:59.157 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:43:59.152+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:43:59.182 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:43:59.179+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:43:59.604 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:43:59.600+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:43:59.663 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:43:59.660+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:43:59.720 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:43:59.716+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:00.006 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:00.003+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:00.144 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:00.140+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:00.238 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:00.234+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:00.620 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:00.616+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:00.660 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:00.657+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:00.758 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:00.754+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:00.837 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:44:00.832+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e580 v580) 2019-12-09T20:44:00.962 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:00.958+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:01.196 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:01.192+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:01.250 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:01.247+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:01.623 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:01.617+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:01.627 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:01.623+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:01.804 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:01.800+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:01.983 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:01.979+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:02.249 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:02.244+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:02.253 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:02.248+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:02.597 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:02.594+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:02.608 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:02.604+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:02.815 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:02.811+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:02.949 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:02.946+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:03.287 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:03.283+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:03.293 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:03.290+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:03.576 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:03.572+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:03.615 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:03.611+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:03.801 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:03.794+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:04.000 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:03.998+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:04.258 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:04.255+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:04.321 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:04.317+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:04.578 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:04.575+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:04.580 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:04.575+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:04.794 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:04.790+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:05.032 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:05.027+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:05.239 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:05.236+0000 1694f700 -1 osd.1 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:37.912302+0000 front 2019-12-09T20:42:37.948755+0000 (oldest deadline 2019-12-09T20:43:23.203088+0000) 2019-12-09T20:44:05.279 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:44:05.275+0000 1694f700 -1 osd.6 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:39.450694+0000 front 2019-12-09T20:42:39.669921+0000 (oldest deadline 2019-12-09T20:43:23.381380+0000) 2019-12-09T20:44:05.603 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:44:05.599+0000 1694f700 -1 osd.0 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.286973+0000 front 2019-12-09T20:42:38.174460+0000 (oldest deadline 2019-12-09T20:43:47.771691+0000) 2019-12-09T20:44:05.620 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:44:05.608+0000 1694f700 -1 osd.7 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:41.335896+0000 front 2019-12-09T20:42:41.337745+0000 (oldest deadline 2019-12-09T20:43:23.561677+0000) 2019-12-09T20:44:05.789 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:44:05.785+0000 1694f700 -1 osd.5 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:40.496428+0000 front 2019-12-09T20:42:40.598478+0000 (oldest deadline 2019-12-09T20:43:25.720445+0000) 2019-12-09T20:44:05.921 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:44:05.910+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e580 v580) 2019-12-09T20:44:06.022 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:44:06.018+0000 1694f700 -1 osd.2 580 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:42:38.930052+0000 front 2019-12-09T20:42:39.005011+0000 (oldest deadline 2019-12-09T20:43:23.187166+0000) 2019-12-09T20:44:49.792 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:49.560+0000 33389700 -1 osd.1 584 build_incremental_map_msg missing incremental map 584 2019-12-09T20:44:49.824 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:49.563+0000 11caf700 -1 osd.1 584 build_incremental_map_msg missing incremental map 584 2019-12-09T20:44:49.888 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:49.618+0000 32b88700 -1 osd.1 584 build_incremental_map_msg missing incremental map 584 2019-12-09T20:44:49.949 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:44:49.820+0000 38393700 -1 osd.1 584 build_incremental_map_msg missing incremental map 584 2019-12-09T20:45:21.607 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:24:13.458 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.611 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:24:13.462 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.661 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:24:13.486 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.666 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:24:13.491 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.687 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:24:13.512 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.693 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:24:13.519 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.825 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:24:13.674 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.882 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:24:13.708 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:21.945 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:24:13.771 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:45:54.616 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:54.612+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:54.703 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:54.700+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:55.620 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:55.615+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:55.660 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:55.657+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:56.330 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:45:56.326+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:45:56.643 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:56.639+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:56.659 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:56.657+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:57.362 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:45:57.359+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:45:57.621 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:57.617+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:57.687 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:57.683+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:57.862 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:45:57.859+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:45:58.328 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:45:58.324+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:45:58.414 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:45:58.410+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:45:58.589 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:58.586+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:58.662 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:58.658+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:58.872 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:45:58.868+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:45:59.303 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:45:59.293+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:45:59.418 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:45:59.414+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:45:59.599 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:45:59.596+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:45:59.619 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:45:59.615+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:45:59.857 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:45:59.853+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:00.119 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:00.114+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:00.296 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:00.292+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:00.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:00.399+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:00.566 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:00.563+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:00.632 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:00.628+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:00.867 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:00.864+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:01.075 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:01.072+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:01.328 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:01.324+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:01.405 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:01.401+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:01.623 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:01.619+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:01.687 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:01.683+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:01.840 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:01.838+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:02.053 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:02.050+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:02.380 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:02.374+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:02.430 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:02.425+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:02.656 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:02.652+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:02.669 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:02.664+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:02.856 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:02.851+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:03.079 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:03.076+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:03.336 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:03.332+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:03.423 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:03.419+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:03.693 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:03.689+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:03.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:03.699+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:03.810 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:03.805+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:04.041 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:04.036+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:04.332 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:04.328+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:04.424 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:04.420+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:04.726 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:04.722+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:04.753 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:04.745+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:04.773 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:04.768+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:05.085 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:05.080+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:05.301 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:05.298+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:05.447 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:05.439+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:05.693 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:05.689+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:05.737 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:05.732+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:05.802 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:05.798+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:06.144 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:06.139+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:06.300 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:06.295+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:06.480 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:06.473+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:06.707 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:06.701+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:06.719 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:06.715+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:06.831 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:06.827+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:07.166 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:07.162+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:07.294 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:07.291+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:07.498 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:07.494+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:07.678 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:07.673+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:07.740 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:07.736+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:07.850 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:07.846+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:08.150 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:08.146+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:08.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:08.313+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:08.528 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:08.523+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:08.641 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:08.637+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:08.793 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:08.789+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:08.887 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:08.883+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:09.125 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:09.121+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:09.274 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:09.270+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:09.554 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:09.549+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:09.690 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:09.686+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:09.764 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:09.760+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:09.865 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:09.860+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:10.137 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:10.133+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:10.261 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:10.257+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:10.535 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:10.532+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:10.695 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:10.690+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:10.725 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:10.721+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:10.843 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:10.840+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:11.155 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:11.146+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:11.214 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:11.210+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:11.554 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:11.541+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:11.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:11.685+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:11.694 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:11.689+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:11.859 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:11.855+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:12.168 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:12.162+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:12.257 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:12.253+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:12.596 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:12.592+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:12.680 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:12.676+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:12.706 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:12.703+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:12.884 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:12.879+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:13.135 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:13.130+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:13.297 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:13.292+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:13.550 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:13.547+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:13.679 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:13.674+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:13.696 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:13.692+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:13.853 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:13.849+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:14.184 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:14.179+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:14.313 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:14.308+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:14.541 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:14.537+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:14.669 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:14.665+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:14.710 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:14.706+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:14.845 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:14.841+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:15.162 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:15.157+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:15.276 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:15.272+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:15.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:15.538+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:15.690 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:15.686+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:15.698 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:15.693+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:15.858 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:15.853+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:16.165 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:16.162+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:16.249 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:16.246+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:16.501 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:16.497+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:16.712 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:16.708+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:16.739 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:16.734+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:16.846 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:16.840+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:17.205 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:17.200+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:17.210 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:17.205+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:17.499 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:17.495+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:17.730 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:17.726+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:17.779 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:17.773+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:17.883 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:17.877+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:18.197 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:18.194+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:18.255 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:18.251+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:18.471 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:18.467+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:18.760 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:18.755+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:18.795 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:18.791+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:18.859 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:18.855+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:19.158 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:19.154+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:19.239 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:19.235+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:19.516 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:19.513+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:19.715 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:19.711+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:19.793 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:19.788+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:19.835 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:19.830+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:20.129 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:20.124+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:20.238 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:20.233+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:20.566 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:20.562+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:20.717 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:20.713+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:20.818 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:20.813+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:20.871 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:20.866+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:21.156 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:21.148+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:21.205 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:21.202+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:21.524 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:21.520+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:21.689 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:21.685+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:21.811 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:21.807+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:21.862 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:21.858+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:22.182 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:22.178+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:22.218 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:22.214+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:22.551 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:22.547+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:22.652 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:22.649+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:22.821 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:22.815+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:22.877 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:22.872+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:23.219 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:23.214+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:23.223 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:23.219+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:23.530 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:23.525+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:23.637 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:23.633+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:23.850 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:23.846+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:23.860 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:23.855+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:24.228 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:24.223+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:24.265 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:24.260+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:24.551 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:24.547+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:24.592 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:24.589+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:24.846 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:24.841+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:24.850 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:24.845+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:25.223 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:25.219+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:25.266 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:25.261+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:25.560 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:25.556+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:25.584 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:25.579+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:25.814 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:25.809+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:25.826 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:25.822+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:26.241 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:26.220+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:26.268 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:26.263+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:26.546 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:26.540+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:26.600 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:26.595+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:26.781 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:46:26.777+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e588 v588) 2019-12-09T20:46:26.830 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:26.826+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:26.866 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:26.861+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:27.246 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:27.242+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:27.274 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:27.271+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:27.587 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:27.577+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:27.599 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:27.593+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:27.846 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:27.842+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:27.856 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:27.852+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:28.251 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:28.247+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:28.318 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:28.315+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:28.582 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:28.569+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:28.610 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:28.607+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:28.798 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:28.795+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:28.888 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:28.884+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:29.250 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:29.246+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:29.284 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:29.281+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:29.557 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:29.554+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:29.604 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:29.599+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:29.844 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:29.839+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:29.915 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:29.913+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:30.225 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:30.223+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:30.260 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:30.257+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:30.554 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:30.552+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:30.648 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:30.645+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:30.875 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:30.872+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:30.911 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:30.907+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:31.226 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:31.223+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:31.290 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:31.282+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:31.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:31.539+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:31.649 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:31.646+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:31.806 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:46:31.802+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e588 v588) 2019-12-09T20:46:31.851 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:31.847+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:31.927 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:31.924+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:32.264 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:32.260+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:32.281 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:32.278+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:32.542 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:32.537+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:32.642 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:32.638+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:32.859 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:32.855+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:32.942 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:32.940+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:33.222 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:33.218+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:33.319 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:33.316+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:33.513 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:33.509+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:33.683 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:33.679+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:33.854 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:33.850+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:33.947 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:33.944+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:34.256 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:34.251+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:34.281 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:34.278+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:34.552 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:34.548+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:34.659 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:34.656+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:34.887 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:34.884+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:34.917 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:34.914+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:35.300 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:35.296+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:35.308 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:35.304+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:35.546 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:35.541+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:35.658 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:35.654+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:35.938 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:35.935+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:35.941 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:35.937+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:46:36.302 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:46:36.299+0000 1694f700 -1 osd.1 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.950331+0000 front 2019-12-09T20:45:11.951452+0000 (oldest deadline 2019-12-09T20:45:55.880094+0000) 2019-12-09T20:46:36.347 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:46:36.343+0000 1694f700 -1 osd.7 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:15.529966+0000 front 2019-12-09T20:45:13.846402+0000 (oldest deadline 2019-12-09T20:45:59.420859+0000) 2019-12-09T20:46:36.589 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:46:36.585+0000 1694f700 -1 osd.6 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:13.135522+0000 front 2019-12-09T20:45:13.282428+0000 (oldest deadline 2019-12-09T20:45:58.287042+0000) 2019-12-09T20:46:36.691 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:46:36.687+0000 1694f700 -1 osd.5 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:11.823902+0000 front 2019-12-09T20:45:11.277312+0000 (oldest deadline 2019-12-09T20:45:54.413624+0000) 2019-12-09T20:46:36.895 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:46:36.892+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 45sec e588 v588) 2019-12-09T20:46:36.969 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:46:36.965+0000 1694f700 -1 osd.0 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:12.941119+0000 front 2019-12-09T20:45:12.296321+0000 (oldest deadline 2019-12-09T20:45:57.570596+0000) 2019-12-09T20:46:36.970 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:46:36.966+0000 1694f700 -1 osd.2 588 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:45:10.018652+0000 front 2019-12-09T20:45:09.864323+0000 (oldest deadline 2019-12-09T20:45:53.782381+0000) 2019-12-09T20:47:07.371 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:25:59.222 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.382 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:25:59.207 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.387 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:25:59.212 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.390 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:25:59.216 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.395 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:25:59.221 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.408 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:25:59.259 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.501 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:25:59.327 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.557 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:25:59.383 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:07.589 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:25:59.440 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:47:23.140 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:47:23.125+0000 36f91700 -1 osd.3 593 build_incremental_map_msg missing incremental map 593 2019-12-09T20:47:25.155 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:47:25.144+0000 32f89700 -1 osd.3 594 build_incremental_map_msg missing incremental map 593 2019-12-09T20:48:21.915 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:21.911+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:22.949 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:22.944+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:22.949 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:22.945+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:23.214 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:23.211+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:23.957 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:23.953+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:23.964 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:23.960+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:24.002 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:23.997+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:24.253 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:24.249+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:24.265 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:24.262+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:24.964 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:24.959+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:24.999 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:24.994+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:25.001 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:24.997+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:25.256 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:25.253+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:25.301 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:25.298+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:25.982 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:25.975+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:26.037 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:26.034+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:26.049 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:26.045+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:26.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:26.224+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:26.351 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:26.348+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:26.943 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:26.939+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:26.965 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:26.961+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:27.001 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:26.997+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:27.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:27.040+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:27.249 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:27.246+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:27.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:27.397+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:27.904 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:27.897+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:27.954 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:27.950+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:27.985 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:27.982+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:28.066 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:28.061+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:28.284 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:28.281+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:28.410 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:28.406+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:28.919 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:28.916+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:28.921 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:28.918+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:28.959 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:28.953+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:29.019 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:29.015+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:29.262 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:29.258+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:29.373 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:29.369+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:29.889 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:29.885+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:29.916 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:29.912+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:29.919 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:29.915+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:30.069 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:30.066+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:30.277 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:30.273+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:30.387 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:30.384+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:30.847 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:30.844+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:30.904 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:30.900+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:30.959 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:30.956+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:31.103 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:31.098+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:31.263 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:31.259+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:31.426 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:31.422+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:31.808 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:31.803+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:31.894 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:31.887+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:31.981 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:31.978+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:32.072 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:32.068+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:32.240 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:32.235+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:32.416 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:32.413+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:32.813 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:32.808+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:32.859 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:32.855+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:33.011 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:33.007+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:33.031 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:33.027+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:33.281 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:33.276+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:33.419 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:33.416+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:33.833 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:33.828+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:33.846 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:33.842+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:33.988 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:33.984+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:34.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:34.024+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:34.266 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:34.262+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:34.455 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:34.450+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:34.790 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:34.786+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:34.867 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:34.864+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:34.991 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:34.986+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:35.027 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:35.023+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:35.237 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:35.230+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:35.436 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:35.432+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:35.756 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:35.752+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:35.918 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:35.914+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:36.020 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:36.015+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:36.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:36.025+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:36.244 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:36.240+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:36.466 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:36.462+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:36.770 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:36.767+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:36.974 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:36.970+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:37.026 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:37.023+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:37.049 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:37.044+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:37.227 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:37.223+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:37.446 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:37.442+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:37.777 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:37.775+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:38.003 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:38.001+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:38.004 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:38.001+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:38.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:38.012+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:38.248 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:38.244+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:38.501 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:38.494+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:38.758 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:38.754+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:38.986 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:38.982+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:39.022 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:39.017+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:39.051 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:39.047+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:39.278 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:39.274+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:39.470 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:39.466+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:39.721 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:39.717+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:40.005 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:40.000+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:40.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:40.009+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:40.029 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:40.011+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:40.326 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:40.321+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:40.452 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:40.447+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:40.752 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:40.748+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:40.964 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:40.960+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:41.001 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:40.997+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:41.063 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:41.059+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:41.294 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:41.290+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:41.465 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:41.460+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:41.740 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:41.736+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:41.930 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:41.926+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:42.013 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:42.009+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:42.028 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:42.023+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:42.319 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:42.315+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:42.479 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:42.474+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:42.739 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:42.735+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:42.971 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:42.966+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:42.985 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:42.980+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:43.017 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:43.012+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:43.345 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:43.341+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:43.519 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:43.515+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:43.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:43.784+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:43.985 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:43.981+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:43.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:43.990+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:44.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:44.011+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:44.339 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:44.335+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:44.491 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:44.486+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:44.786 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:44.783+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:44.966 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:44.961+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:44.972 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:44.968+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:44.994 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:44.991+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:45.296 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:45.292+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:45.504 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:45.500+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:45.816 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:45.812+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:45.960 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:45.956+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:46.007 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:46.003+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:46.023 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:46.019+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:46.333 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:46.330+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:46.485 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:46.480+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:46.799 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:46.795+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:46.972 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:46.968+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:46.989 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:46.985+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:47.038 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:47.034+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:47.367 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:47.363+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:47.444 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:47.440+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:47.827 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:47.822+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:47.947 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:47.942+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:47.950 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:47.946+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:48.033 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:48.028+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:48.336 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:48.333+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:48.445 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:48.441+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:48.844 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:48.840+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:48.918 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:48.915+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:48.937 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:48.934+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:49.019 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:49.015+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:49.312 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:49.307+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:49.494 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:49.489+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:49.820 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:49.816+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:49.964 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:49.960+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:49.991 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:49.986+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:50.016 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:50.012+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:50.288 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:50.284+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:50.449 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:50.445+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:50.809 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:50.806+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:50.948 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:50.944+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:50.974 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:50.970+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:51.039 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:51.034+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:51.317 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:51.313+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:51.406 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:51.402+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:51.831 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:51.826+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:51.912 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:51.908+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:51.949 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:51.943+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:52.042 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:52.039+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:52.331 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:52.328+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:52.434 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:52.429+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:52.860 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:52.856+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:52.958 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:52.954+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:52.979 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:52.974+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:53.029 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:53.024+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:53.324 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:53.320+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:53.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:53.433+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:53.818 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:53.814+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:53.959 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:53.956+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:54.009 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:54.005+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:54.037 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:54.031+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:54.292 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:54.288+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:54.448 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:54.444+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:54.796 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:54.792+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:54.971 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:54.967+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:55.044 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:55.039+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:55.051 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:55.047+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:55.338 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:55.334+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:55.437 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:55.433+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:55.767 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:55.763+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:55.993 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:55.988+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:56.005 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:56.000+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:56.020 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:56.016+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:56.387 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:56.377+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:56.421 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:56.417+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:56.770 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:56.766+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:57.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:56.994+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:57.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:57.040+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:57.058 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:57.053+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:57.416 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:57.412+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:57.420 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:57.416+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:57.538 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:48:57.534+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e595 v595) 2019-12-09T20:48:57.795 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:57.791+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:57.957 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:57.953+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:58.027 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:58.024+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:58.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:58.040+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:58.441 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:58.438+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:58.445 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:58.442+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:58.772 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:58.769+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:48:59.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:48:59.003+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:48:59.027 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:48:59.024+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:48:59.074 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:48:59.069+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:48:59.443 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:48:59.440+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:48:59.475 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:48:59.471+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:48:59.794 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:48:59.790+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:00.027 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:00.023+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:00.043 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:00.040+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:00.044 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:00.040+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:00.466 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:00.463+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:00.506 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:00.503+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:00.810 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:00.806+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:01.007 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:00.998+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:01.007 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:00.996+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:01.084 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:01.080+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:01.440 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:01.435+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:01.526 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:01.523+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:01.827 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:01.822+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:01.986 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:01.982+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:01.996 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:01.993+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:02.121 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:02.117+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:02.415 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:02.411+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:02.549 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:02.546+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:02.560 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:49:02.555+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e595 v595) 2019-12-09T20:49:02.800 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:02.794+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:02.980 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:02.976+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:03.000 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:02.997+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:03.152 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:03.148+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:03.377 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:03.373+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:03.573 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:03.569+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:03.768 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:03.764+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:03.947 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:03.943+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:04.047 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:04.044+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:04.138 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:04.135+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:04.431 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:04.427+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:04.551 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:04.547+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:04.788 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:04.784+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:04.926 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:04.921+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:05.064 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:05.060+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:05.137 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:05.133+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:05.400 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:05.393+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:05.601 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:05.596+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:05.833 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:05.830+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:05.927 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:05.923+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:06.053 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:06.050+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:06.105 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:06.100+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:06.368 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:06.364+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:06.651 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:06.647+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:06.787 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:06.784+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:06.901 INFO:tasks.ceph.osd.7.smithi106.stderr:2019-12-09T20:49:06.897+0000 1694f700 -1 osd.7 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:40.852144+0000 front 2019-12-09T20:47:40.583671+0000 (oldest deadline 2019-12-09T20:48:23.982375+0000) 2019-12-09T20:49:07.044 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:49:07.040+0000 1694f700 -1 osd.1 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:36.313105+0000 front 2019-12-09T20:47:36.311757+0000 (oldest deadline 2019-12-09T20:48:22.125606+0000) 2019-12-09T20:49:07.089 INFO:tasks.ceph.osd.6.smithi106.stderr:2019-12-09T20:49:07.085+0000 1694f700 -1 osd.6 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.388552+0000 front 2019-12-09T20:47:41.641090+0000 (oldest deadline 2019-12-09T20:48:26.029500+0000) 2019-12-09T20:49:07.346 INFO:tasks.ceph.osd.0.smithi174.stderr:2019-12-09T20:49:07.343+0000 1694f700 -1 osd.0 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.172877+0000 front 2019-12-09T20:47:41.032988+0000 (oldest deadline 2019-12-09T20:48:23.770804+0000) 2019-12-09T20:49:07.678 INFO:tasks.ceph.mon.b.smithi106.stderr:2019-12-09T20:49:07.669+0000 1a89c700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.174:6823/10740,v1:172.21.15.174:6825/10740] for 47sec e595 v595) 2019-12-09T20:49:07.704 INFO:tasks.ceph.osd.2.smithi174.stderr:2019-12-09T20:49:07.700+0000 1694f700 -1 osd.2 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:37.389054+0000 front 2019-12-09T20:47:37.529556+0000 (oldest deadline 2019-12-09T20:48:23.184598+0000) 2019-12-09T20:49:07.766 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:49:07.762+0000 1694f700 -1 osd.5 595 heartbeat_check: no reply from 172.21.15.174:6828 osd.3 since back 2019-12-09T20:47:41.396440+0000 front 2019-12-09T20:47:41.826967+0000 (oldest deadline 2019-12-09T20:48:21.633606+0000) 2019-12-09T20:49:49.931 INFO:tasks.ceph.osd.3.smithi174.stderr:*** Caught signal (Segmentation fault) ** 2019-12-09T20:49:49.931 INFO:tasks.ceph.osd.3.smithi174.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:49:50.014 INFO:tasks.ceph.osd.3.smithi174.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:49:50.014 INFO:tasks.ceph.osd.3.smithi174.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:49:50.014 INFO:tasks.ceph.osd.3.smithi174.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:49:50.014 INFO:tasks.ceph.osd.3.smithi174.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:49:50.015 INFO:tasks.ceph.osd.3.smithi174.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:49:50.016 INFO:tasks.ceph.osd.3.smithi174.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:49:50.016 INFO:tasks.ceph.osd.3.smithi174.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:49:50.016 INFO:tasks.ceph.osd.3.smithi174.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:49:50.079 INFO:tasks.ceph.osd.3.smithi174.stderr:2019-12-09T20:49:50.073+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:49:50.079 INFO:tasks.ceph.osd.3.smithi174.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:49:50.079 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:49:50.080 INFO:tasks.ceph.osd.3.smithi174.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:49:50.081 INFO:tasks.ceph.osd.3.smithi174.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:49:50.081 INFO:tasks.ceph.osd.3.smithi174.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:49:50.081 INFO:tasks.ceph.osd.3.smithi174.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:49:50.081 INFO:tasks.ceph.osd.3.smithi174.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:49:50.082 INFO:tasks.ceph.osd.3.smithi174.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:49:50.082 INFO:tasks.ceph.osd.3.smithi174.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:49:50.082 INFO:tasks.ceph.osd.3.smithi174.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:49:50.082 INFO:tasks.ceph.osd.3.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:49:50.082 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:50.537 INFO:tasks.ceph.osd.3.smithi174.stderr: -26> 2019-12-09T20:49:50.073+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:49:50.537 INFO:tasks.ceph.osd.3.smithi174.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:49:50.537 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:50.537 INFO:tasks.ceph.osd.3.smithi174.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:49:50.538 INFO:tasks.ceph.osd.3.smithi174.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:49:50.539 INFO:tasks.ceph.osd.3.smithi174.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:49:50.540 INFO:tasks.ceph.osd.3.smithi174.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:49:50.540 INFO:tasks.ceph.osd.3.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:49:50.540 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:51.093 INFO:tasks.ceph.osd.3.smithi174.stderr: -1027> 2019-12-09T20:49:50.073+0000 11caf700 -1 *** Caught signal (Segmentation fault) ** 2019-12-09T20:49:51.094 INFO:tasks.ceph.osd.3.smithi174.stderr: in thread 11caf700 thread_name:msgr-worker-1 2019-12-09T20:49:51.094 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:51.094 INFO:tasks.ceph.osd.3.smithi174.stderr: ceph version 15.0.0-7900-g1351c81 (1351c819e1a9518cde4c8b12d645782009028360) octopus (dev) 2019-12-09T20:49:51.094 INFO:tasks.ceph.osd.3.smithi174.stderr: 1: (()+0xf5f0) [0xc09e5f0] 2019-12-09T20:49:51.094 INFO:tasks.ceph.osd.3.smithi174.stderr: 2: (MonClient::handle_auth_done(Connection*, AuthConnectionMeta*, unsigned long, unsigned int, ceph::buffer::v14_2_0::list const&, CryptoKey*, std::string*)+0x71) [0xe8a9b1] 2019-12-09T20:49:51.095 INFO:tasks.ceph.osd.3.smithi174.stderr: 3: (ProtocolV2::handle_auth_done(ceph::buffer::v14_2_0::list&)+0x2e0) [0x100d020] 2019-12-09T20:49:51.095 INFO:tasks.ceph.osd.3.smithi174.stderr: 4: (ProtocolV2::handle_frame_payload()+0x1d3) [0x1019dc3] 2019-12-09T20:49:51.095 INFO:tasks.ceph.osd.3.smithi174.stderr: 5: (ProtocolV2::handle_read_frame_dispatch()+0x150) [0x101a040] 2019-12-09T20:49:51.095 INFO:tasks.ceph.osd.3.smithi174.stderr: 6: (ProtocolV2::handle_read_frame_epilogue_main(std::unique_ptr&&, int)+0x44d) [0x101a59d] 2019-12-09T20:49:51.095 INFO:tasks.ceph.osd.3.smithi174.stderr: 7: (ProtocolV2::run_continuation(Ct&)+0x34) [0x1003224] 2019-12-09T20:49:51.096 INFO:tasks.ceph.osd.3.smithi174.stderr: 8: (AsyncConnection::process()+0x794) [0xfdd874] 2019-12-09T20:49:51.096 INFO:tasks.ceph.osd.3.smithi174.stderr: 9: (EventCenter::process_events(unsigned int, std::chrono::duration >*)+0xa25) [0xe3dd55] 2019-12-09T20:49:51.096 INFO:tasks.ceph.osd.3.smithi174.stderr: 10: (()+0xd3b767) [0xe43767] 2019-12-09T20:49:51.096 INFO:tasks.ceph.osd.3.smithi174.stderr: 11: (()+0x121504f) [0x131d04f] 2019-12-09T20:49:51.096 INFO:tasks.ceph.osd.3.smithi174.stderr: 12: (()+0x7e65) [0xc096e65] 2019-12-09T20:49:51.097 INFO:tasks.ceph.osd.3.smithi174.stderr: 13: (clone()+0x6d) [0xd20e88d] 2019-12-09T20:49:51.097 INFO:tasks.ceph.osd.3.smithi174.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2019-12-09T20:49:51.097 INFO:tasks.ceph.osd.3.smithi174.stderr: 2019-12-09T20:49:54.640 INFO:tasks.ceph.osd.3.smithi174.stderr:daemon-helper: command crashed with signal 11 2019-12-09T20:51:18.473 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:10.324 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.483 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:10.334 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.556 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:10.382 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.558 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:10.383 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.559 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:10.385 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.560 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:10.386 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.743 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:10.569 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.745 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:10.571 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:18.822 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:10.673 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.465 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:25.316 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.473 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:25.324 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.477 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:25.302 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.481 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:25.306 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.483 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:25.309 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.486 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:25.311 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.518 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:30:25.343 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.525 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:30:25.350 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:51:33.525 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:30:25.376 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.957 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:35:38.807 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.978 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:35:38.794 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.978 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:35:38.803 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.979 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:35:38.816 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.995 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:35:38.821 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:46.998 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:35:38.824 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:47.211 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:35:39.037 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:47.246 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:35:39.071 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:56:47.289 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:35:39.140 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.641 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:38:32.492 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.648 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:38:32.499 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.699 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:38:32.524 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.700 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:38:32.525 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.720 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:38:32.545 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.722 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:38:32.548 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.797 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:38:32.623 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.827 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:38:32.678 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:40.879 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:38:32.704 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T20:59:51.479 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:59:51.470+0000 32b88700 -1 osd.1 599 build_incremental_map_msg missing incremental map 561 2019-12-09T20:59:51.502 INFO:tasks.ceph.osd.1.smithi174.stderr:2019-12-09T20:59:51.471+0000 32b88700 -1 osd.1 599 build_incremental_map_msg missing incremental map 584 2019-12-09T20:59:57.624 INFO:tasks.ceph.osd.5.smithi106.stderr:2019-12-09T20:59:57.618+0000 35f8f700 -1 osd.5 603 build_incremental_map_msg missing incremental map 569 2019-12-09T21:00:06.704 INFO:tasks.rgw.client.0.smithi174.stdout:2019-12-09T21:00:06.677+0000 20e25700 -1 RGWWatcher::handle_error cookie 1542261552 err (110) Connection timed out 2019-12-09T21:00:39.487 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:39:31.338 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.492 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:39:31.343 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.502 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:39:31.327 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.505 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:39:31.331 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.507 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:39:31.333 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.511 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:39:31.337 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.669 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:39:31.520 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.671 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:39:31.497 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:00:39.692 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:39:31.517 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.725 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:44:16.572 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.733 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:44:16.584 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.742 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:44:16.567 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.742 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:44:16.567 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.745 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:44:16.571 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.747 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:44:16.573 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.949 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:44:16.775 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.955 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:44:16.781 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:05:24.978 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:44:16.829 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.813 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:46:54.663 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.821 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:46:54.646 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.821 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:46:54.666 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.861 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:46:54.687 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.865 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:46:54.691 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.879 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:46:54.705 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.985 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:46:54.811 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:02.988 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:46:54.840 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:08:03.060 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:46:54.886 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.008 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:26.859 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.024 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:26.876 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.028 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:26.853 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.030 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:26.856 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.031 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:26.856 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.033 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:26.859 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.140 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:26.966 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.142 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:26.968 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:35.222 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:27.073 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.643 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:35.495 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.647 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:35.497 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.669 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:49:35.521 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.703 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:35.529 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.706 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:35.532 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.715 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:35.542 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.720 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:35.546 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.734 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:49:35.561 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:10:43.738 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:49:35.565 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.239 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:54:43.091 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.258 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:54:43.107 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.258 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:54:43.085 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.261 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:54:43.087 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.261 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:54:43.088 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.264 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:54:43.091 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.455 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:54:43.282 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.461 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:54:43.287 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:15:51.530 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:54:43.381 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.122 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:55:21.974 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.130 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:55:21.981 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.146 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:55:21.973 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.168 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:55:21.994 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.199 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:55:22.050 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.201 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:55:22.027 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.228 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:55:22.043 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.243 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:55:22.069 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:16:30.269 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:55:22.095 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.448 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:59:55.299 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.457 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:59:55.308 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.469 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:59:55.295 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.473 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:59:55.300 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.481 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:59:55.307 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.487 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:59:55.314 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.650 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:09:59:55.477 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.651 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:09:59:55.477 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:21:03.702 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:09:59:55.554 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.176 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:03:44.028 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.184 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:03:44.037 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.243 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:03:44.069 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.247 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:03:44.074 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.247 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:03:44.074 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.254 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:03:44.081 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.332 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:03:44.184 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.379 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:03:44.205 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:24:52.396 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:03:44.223 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.733 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:05:05.585 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.746 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:05:05.599 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.752 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:05:05.579 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.756 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:05:05.583 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.757 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:05:05.584 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.761 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:05:05.588 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.827 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:05:05.653 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.854 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:05:05.680 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:26:13.878 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:05:05.730 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.012 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:10:17.864 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.026 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:10:17.878 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.033 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:10:17.860 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.034 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:10:17.860 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.036 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:10:17.863 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.038 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:10:17.865 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.233 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:10:18.060 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.234 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:10:18.061 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:31:26.367 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:10:18.219 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.189 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:12:06.041 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.213 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:12:06.049 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.267 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:12:06.094 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.271 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:12:06.096 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.276 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:12:06.103 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.279 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:12:06.105 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.329 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:12:06.181 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.349 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:12:06.176 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:33:14.374 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:12:06.201 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.196 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:15:28.048 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.207 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:15:28.059 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.219 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:15:28.045 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.221 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:15:28.048 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.225 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:15:28.052 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.226 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:15:28.054 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.375 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:15:28.202 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.386 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:15:28.238 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:36:36.404 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:15:28.231 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.364 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:28.216 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.370 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:28.222 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.389 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:28.212 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.397 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:28.224 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.413 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:28.240 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.441 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:28.268 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.544 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:28.371 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.558 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:28.408 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:36.588 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:28.415 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.528 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:40.381 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.548 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:40.400 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.550 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:40.377 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.550 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:40.377 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.556 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:40.382 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.563 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:40.389 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.601 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:20:40.428 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.602 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:20:40.429 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:41:48.613 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:20:40.465 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.003 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:25:51.855 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.009 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:25:51.861 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.018 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:25:51.845 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.018 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:25:51.845 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.021 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:25:51.848 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.026 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:25:51.852 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.224 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:25:52.050 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.225 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:25:52.052 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:47:00.240 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:25:52.092 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.322 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:28:50.174 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.334 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:28:50.186 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.384 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:28:50.210 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.396 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:28:50.223 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.403 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:28:50.230 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.405 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:28:50.231 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.442 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:28:50.294 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.518 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:28:50.344 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:49:58.523 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:28:50.350 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.001 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:30:25.853 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.015 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:30:25.859 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.067 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:30:25.893 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.068 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:30:25.895 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.088 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:30:25.914 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.092 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:30:25.918 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.101 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:30:25.953 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.143 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:30:25.970 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:51:34.162 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:30:25.988 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.345 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:31:02.195 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.353 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:31:02.203 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.367 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:31:02.193 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.368 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:31:02.194 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.372 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:31:02.199 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.373 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:31:02.199 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.440 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:31:02.267 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.442 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:31:02.268 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:52:10.443 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:31:02.296 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.435 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:36:15.285 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.444 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:36:15.296 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.452 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:36:15.279 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.455 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:36:15.282 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.464 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:36:15.290 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.485 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:36:15.301 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.658 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:36:15.485 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.709 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:36:15.536 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:57:23.768 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:36:15.620 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.736 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:37:12.587 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.749 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:37:12.600 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.780 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:37:12.606 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.811 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:37:12.637 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.818 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:37:12.670 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.827 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:37:12.654 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.832 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:37:12.659 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.881 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:37:12.707 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T21:58:20.892 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:37:12.719 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.675 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:41:27.526 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.698 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:41:27.550 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.703 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:41:27.527 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.707 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:41:27.534 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.712 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:41:27.538 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.721 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:41:27.547 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.904 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:41:27.731 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.909 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:41:27.735 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:02:35.922 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:41:27.774 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.027 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:45:34.879 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.100 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:45:34.950 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.155 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:45:34.981 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.164 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:45:34.990 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.177 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:45:35.003 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.197 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:45:35.023 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.293 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:45:35.144 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.314 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:45:35.141 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:06:43.333 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:45:35.160 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.904 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:46:39.755 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.919 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:46:39.745 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.924 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:46:39.750 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.926 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:46:39.778 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.929 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:46:39.756 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.942 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:46:39.769 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:47.984 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:46:39.811 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:48.017 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:46:39.844 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:07:48.106 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:46:39.958 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.819 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:51:51.671 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.829 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:51:51.680 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.845 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:51:51.671 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.848 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:51:51.675 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.851 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:51:51.677 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:12:59.853 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:51:51.679 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:13:00.049 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:51:51.875 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:13:00.088 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:51:51.915 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:13:00.102 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:51:51.954 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.196 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:53:57.047 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.217 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:53:57.069 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.232 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:53:57.058 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.237 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:53:57.063 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.271 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:53:57.098 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.274 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:53:57.101 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.293 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:53:57.145 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.344 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:53:57.170 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:15:05.345 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:53:57.172 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.157 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:57:03.005 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.168 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:57:03.018 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.169 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:57:02.996 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.181 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:57:03.002 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.181 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:57:03.004 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.184 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:57:03.010 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.327 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:10:57:03.153 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.354 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:10:57:03.180 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:18:11.393 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:10:57:03.244 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.385 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:15.237 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.391 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:15.242 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.401 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:15.227 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.404 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:15.231 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.408 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:15.235 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.411 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:15.237 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.603 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:15.430 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.632 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:15.483 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:23.702 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:15.529 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.334 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:19.184 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.347 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:19.198 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.373 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:02:19.225 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.421 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:19.247 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.424 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:19.249 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.426 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:19.253 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.427 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:19.254 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.446 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:02:19.273 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:23:27.449 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:02:19.276 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.546 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:07:27.397 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.560 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:07:27.412 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.565 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:07:27.391 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.570 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:07:27.397 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.572 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:07:27.399 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.574 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:07:27.401 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.787 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:07:27.614 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.814 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:07:27.641 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:28:35.831 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:07:27.683 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.357 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:10:41.209 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.365 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:10:41.217 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.428 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:10:41.254 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.429 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:10:41.255 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.430 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:10:41.257 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.430 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:10:41.257 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.514 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:10:41.340 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.522 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:10:41.348 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:31:49.553 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:10:41.405 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.143 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:06.995 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.146 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:06.998 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.208 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:07.034 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.210 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:07.036 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.234 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:07.085 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.237 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:07.064 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.244 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:07.070 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.297 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:07.123 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:15.313 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:07.139 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.357 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:38.208 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.373 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:38.225 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.376 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:38.202 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.379 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:38.205 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.381 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:38.207 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.387 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:38.213 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.434 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:12:38.260 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.436 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:12:38.262 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:33:46.508 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:12:38.360 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.855 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:17:51.707 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.869 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:17:51.721 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.871 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:17:51.698 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.876 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:17:51.703 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.881 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:17:51.707 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:38:59.883 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:17:51.710 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:39:00.070 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:17:51.895 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:39:00.072 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:17:51.899 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:39:00.101 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:17:51.953 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.808 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:19:03.660 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.818 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:19:03.669 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.896 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:19:03.722 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.898 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:19:03.724 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.900 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:19:03.752 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.901 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:19:03.727 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.916 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:19:03.742 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.946 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:19:03.772 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:40:11.966 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:19:03.792 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:10.986 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:23:02.837 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:10.998 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:23:02.850 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.009 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:23:02.835 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.012 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:23:02.839 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.013 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:23:02.839 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.020 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:23:02.847 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.162 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:23:02.988 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.182 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:23:03.009 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:44:11.206 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:23:03.058 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:33.982 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:27:25.833 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:33.986 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:27:25.838 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.006 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:27:25.832 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.009 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:27:25.835 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.044 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:27:25.870 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.046 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:27:25.873 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.160 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:27:25.986 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.165 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:27:25.991 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:48:34.220 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:27:26.072 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.402 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:28:13.254 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.421 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:28:13.273 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.436 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:28:13.262 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.439 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:28:13.265 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.441 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:28:13.267 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.454 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:28:13.280 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.517 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:28:13.368 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.525 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:28:13.351 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:49:21.531 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:28:13.358 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.035 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:33:26.886 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.055 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:33:26.906 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.064 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:33:26.890 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.067 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:33:26.894 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.073 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:33:26.900 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.075 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:33:26.901 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.297 INFO:tasks.ceph.mon.a.smithi174.stderr:==00:11:33:27.123 9280== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.346 INFO:tasks.ceph.mon.b.smithi106.stderr:==00:11:33:27.198 8832== Warning: unimplemented fcntl command: 1036 2019-12-09T22:54:35.382 INFO:tasks.ceph.mon.c.smithi174.stderr:==00:11:33:27.208 9279== Warning: unimplemented fcntl command: 1036 2019-12-09T22:55:27.450 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-12-09T22:55:27.450 DEBUG:teuthology.task.console_log:Killing console logger for smithi106 2019-12-09T22:55:27.450 DEBUG:teuthology.task.console_log:Killing console logger for smithi174 2019-12-09T22:55:27.451 DEBUG:teuthology.task.console_log:Killing console logger for smithi106 2019-12-09T22:55:27.451 DEBUG:teuthology.task.console_log:Killing console logger for smithi174 2019-12-09T22:55:27.451 DEBUG:teuthology.exit:Finished running handlers